jOOQ Tuesdays: Nicolai Parlog Talks About Java 9

Welcome to the jOOQ Tuesdays series. In this series, we’ll publish an article on the third Tuesday every other month where we interview someone we find exciting in our industry from a jOOQ perspective. This includes people who work with SQL, Java, Open Source, and a variety of other related topics.

I’m very excited to feature today Nicolai Parlog, author of The Java Module System

Nicolai, your blog is an “archeological” treasure trove for everyone who wants to learn about why Java expert group decisions were made. What made you dig out all these interesting discussions on the mailing lists?

Ha, thank you, didn’t know I was sitting on a treasure.

It all started with everyone’s favorite bikeshed: Optional. After using it for a few months, I was curious to learn more about the reason behind its introduction to Java and why it was designed the way it was, so I started digging and learned a few things:

  • Piperman, the JDK mailing list archive, is a horrible place to peruse and search.
  • Mailing list discussions are often lengthy, fragmented, and thus hard to revisit.
  • Brian Goetz was absolutely right: Everything related to Optional seems to take 300 messages.

Consequently, researching that post about Optional’s design took a week or so. But as you say, it’s interesting to peek behind the curtain and once a discussion is condensed to its most relevant positions and peppered with some context it really appeals to the wider Java community.

I actually think there’s a niche to be filled, here. Imagine there were a site that did regularly (at least once a week) what I did with a few selected topics: Follow the JDK mailing list, summarize ongoing discussions, and make them accessible to a wide audience. That would be a great service to the Java community as it would make it much easier to follow what is going on and to chime in with an informed opinion when you feel you have something to contribute. Now we just need to find someone with a lot of free time on their hands.

By the way, I think it’s awesome that the comparitively open development of the JDK makes that possible.

I had followed your blog after Java 8 came out, where you explained expert group decisions in retrospect. Now, you’re mostly covering what’s new in Java 9. What are your favourite “hidden” (i.e. non-Jigsaw) Java 9 features and why?

From the few language changes, it’s easy pickings: definitely private interface methods. I’ve been in the situation more than once that I wanted to share code between default methods but found no good place to put it without making it part of the public API. With private mehods in interfaces, that’s a thing of the past.

When it comes to API changes, the decision is much harder as there is more to choose from. People definitely like collection factory methods and I do, too, but I think I’ll go with the changes to Stream and Optional. I really enjoy using those Java 8 features and think it’s great that they’ve been improved in 9.

A JVM feature I really like are multi-release JARs. The ability to ship a JAR that uses the newest APIs, but degrades gracefully on older JVMs will come in very handy. Some projects, Spring for example, already do this, but without JVM support it’s not exactly pleasant.

Can I go on? Because there’s so much more! Just two: Unified logging makes it much easier to tease out JVM log messages without having to configure logging for different subsystems and compact strings and indified string concatenation make working with strings faster, reduce garbage and conserve heap space (on average, 10% to 15% less memory!). Ok, that were three, but there you go.

You’re writing a book on the Java 9 module system that can already be pre-ordered on Manning. What will readers get out of your book?

All they need to become module system experts. Of course it explains all the basics (delcaring, compiling, packaging, and running modular applications) and advanced features (services, implied readability, optional dependencies, etc), but it goes far beyond that. More than how to use a feature it also explains when and why to use it, which nuances to consider, and what are good defaults if you’re not sure which way to go.

It’s also full of practical advice. I migrated two large applications to Java 9 (compiling and running on the new release, not turning them into modules) and that experience as well as the many discussions on the mailing list informed a big chapter on migration. If readers are interested in a preview, I condensed it into a post on the most common Java 9 migration challenges. I also show how to debug modules and the module system with various tools (JDeps for example) and logging (that’s when I started using uniform logging), Last but not least, I plan to include a chapter that simply lists error messages and what to do about them.

In your opinion, what are the good parts and the bad parts about  Jigsaw? Do you think Jigsaw will be adopted quickly?

The good, the bad, and the ugly, eh? My favorite feature (of all of Java 9 actually) is strong encapsulation. The ability to have types that are public only within a module is incredibly valuable! This adds another option to the private-to-public-axis and once people internalize that feature we will wonder how we ever lived without it. Can you imagine giving up private? We will think the same about exported.

I hope the worst aspect of the module system will be the compatibility challenges. That’s a weird way to phrase it, but let me explain. These challenges definitely exist and they will require a non-neglectable investmement from the Java community as a whole to get everything working on Java 9, in the long run as modules. (As an aside: This is well invested time – much of it pays back technical debt.)

My hope is that no other aspect of the module system turns out to be worse. One thing I’m a little concerned about is the strictness of reliable configuration. I like the general principle and I’m definitely one for enforcing good practices, but just think about all those POMs that busily exclude transitive dependencies. Once all those JARs are modules, that won’t work – the module system will not let you launch without all dependencies present.

Generally speaking, the module system makes it harder to go against the maintainers’ decisions. Making internal APIs available via reflection or altering dependencies now goes against the grain of a mechanism that is built deeply into the compiler and JVM. There are of course a number of command line flags to affect the module system but they don’t cover everything. To come back to exclusing dependencies, maybe–ignore-missing-modules ${modules} would be a good idea…

Regarding adoption rate, I expect it to be slower than Java 8. But leaving those projects aside that see every new version as insurmountable and are still on Java 6, I’m sure the vast majority will migrate eventually. If not for Java 9’s features than surely for future ones. As a friend and colleague once said: “I’ll do everything to get to value types.”

Now that Java 9 is out and “legacy”, what Java projects will you cover next in your blog and your work?

Oh boy, I’m still busy with Java 9. First I have to finish the book (November hopefully) and then I want to do a few more migrations because I actually like doing that for some weird and maybe not entirely healthy reason (the things you see…). FYI, I’m for hire, so if readers are stuck with their migration they should reach out.

Beyond that, I’m already looking forward to primitive specialization, e.g. ArrayList<int>, and value types (both from Project Valhalla) as well as the changes Project Amber will bring to Java. I’m sure I’ll start discussing those in 2018.

Another thing I’ll keep myself busy with and which I would love your readers to check out is my YouTube channel. It’s still very young and until the book’s done I won’t do a lot of videos (hope to record one next week), but I’m really thrilled about the whole endavour!

5 Things You May Not Have Known About jOOQ

jOOQ has been around for a while now (since 2009!) and by now we can say we’ve seen quite a bit of things about the SQL and Java languages. Some of our design decisions are particular in the way jOOQ thinks about programming with SQL. These include:

  • Nullability (let’s stop fighting it)
  • Value types (let’s stop pretending SQL has identities)
  • Everything is a table (this really helps get the most out of SQL)
  • Queries are side-effect free functions

jOOQ incorporates all of these ideas. Here are 5 Things You May Not Have Known About jOOQ:

1. Every Column Type is Nullable

SQL NULL is a subtly different beast from Java null, even if programmers often use it for the same thing: Something that is “uninitialised”, some value that we don’t “care about” (yet), or some value that we “don’t need”. A good example would be a middle name:

CREATE TABLE person (
  first_name  VARCHAR(50) NOT NULL,
  middle_name VARCHAR(50),
  last_name   VARCHAR(50) NOT NULL,
  ..
);

Of course, a sufficiently pessimistic programmer will immediately see tons of flaws with the above design. Go read this article about falsehoods programmers believe about names for details.

But anyway, the important thing to understand about NOT NULL constraints in SQL is the fact that they’re… constaints. Just like UNIQUE constraints, FOREIGN KEY constraints, or CHECK constraints.

In fact, they are CHECK constraints. We could rewrite the above table as such:

CREATE TABLE person (
  first_name  VARCHAR(50) CHECK (first_name IS NOT NULL),
  middle_name VARCHAR(50),
  last_name   VARCHAR(50) CHECK (last_name IS NOT NULL),
  ..
);

… and the table would be semantically equivalent. This constraint is just so common that it has a special syntax for it (which is also sometimes better optimised than the equivalent check constraint).

Sidenote: An even more sophisticated constraint type is the SQL standard assertion, which unfortunately hasn’t been implemented in any database I’m aware of yet. There are discussions of adding it to a future Oracle version, though. Assertions are like CHECK constraints, but they work on the entire table / schema / whatever scope. For instance, we could assert that every department of a company must have at least one manager. Currently, we can do this sort of thing only through triggers.

The important message here is that a constraint is a validation on the entire data set (or on a subset, down to an individual row). It is not a type modifier, because even if the NOT NULL constraint may have direct optimisation implications on the column type it is attached to, it is a separate construct that can even be deferred. While languages don’t have to be this way (e.g. Ceylon models constraints directly on types), SQL works like this.

Two examples:

  1. DEFAULT columns: When you have an identity column or some sort of GENERATED BY DEFAULT AS... clause on your column, then the value in the column may be generated by default (duh), which may include – depending on the RDBMS vendor – the generation of a value when it is NULL.
  2. DEFERRED constraints: Some databases (e.g. PostgreSQL) support deferred constraints, i.e. constraints that are validated only when the transaction is committed. This can be specified on individual constraints, or on the session. Which means that the value NULL is a totally acceptable value for a NOT NULL column for a certain amount of time.

Both of the above imply that we must not take NOT NULL as a type modifier, the way some languages have started doing it, like Ceylon or Kotlin:

val a: String? = null;
val b: String = a; // Error

In such languages, String? and String are distinct types, specifically in Ceylon where String? is just syntax sugar for the union type String|Null.

But not in SQL. If a Java API wants to properly reflect the SQL language the way jOOQ does, then all types must be nullable. It is a mistake to:

  • Use primitive types
  • Use Option(al) (there are other caveats with these related to generic type erasure)
  • Use non-null types in languages that have them
  • Use validation annotations (we made that mistake, unfortunately)
  • Use JSR-305 or JSR-308 annotations

Sidenote: If this constraint information should be annotated in Java classes, then JPA @Column(nullable=true) annotations are acceptable, because they simply map to the constraint without any implications on the type. The implications are applied on the persistence behaviour, which is reasonable.

Besides, even if at first, encoding nullability through e.g. Option(al) seems reasonable, it breaks as soon as you outer join anything, e.g.:

SELECT p.*
FROM dual
LEFT JOIN person p
ON p.first_name = 'Ooops, no one by that name'

The above query will produce a single person record with only NULL values in its columns. DESPITE the NOT NULL constraints. Ooops. We’ll get null in non-optional types.

Similar things can happen with unions, grouping sets, functions, and a few other operations.

Takeaway

In SQL, all types are always nullable. We simply have to deal with this. Every clever type safety is contrary to SQL logic. If your API does this, you may get some minor convenience in 80% of the use-cases for the price of a major annoyance in 20% of the use-cases. That’s not a reasonable tradeoff given that in Java, every non-primitive type is nullable as well, so we got a perfect and intuitive match.

2. SQL is a Set-Based, Values-Only Language

Values or Objects? That’s a tricky question for people who work with Java, a language that claims to be mainly object-oriented. Java has value support as well. There are 8 different value types as of Java 8:

  • byte
  • short
  • int
  • long
  • float
  • double
  • boolean
  • char

Values have a couple of nice properties:

  • They are immutable. It may be possible to mutate a variable holding such a value, but we cannot mutate the value itself. 42 will always stay 42
  • Two values that are equal are undistinguishable. 42 == 42 really means that they’re the exact same thing. Reusing == for value equality and identity equality has been a bit of an unfortunate choice in Java, because technically, a String is also a value, yet we cannot compare it with == because there’s a possibility of two identical strings having different identity. (True) values don’t have identity.

Java 8 introduced the notion of a “ValueBased” class, which is really a weird thing, because a “ValueBased” wrapper like Optional can reference a non-value based type, say, a java.sql.Connection. Not a good idea, but certainly possible.

A future Java might have more complex value types, for instance:

// Hypothetical syntax
value Point(int x, int y) {}
value Box(Point a, Point b) {
  int area() {
    return Math.abs(a.x - b.x * a.y - b.y);
  }
}

This will certainly be helpful (as soon as we’ll figure out how to model nullability in such scenarios).

In SQL, all records are values. They do not have a true identity (although most databases choose to provide implementation specific identities like ROWIDs). Do not confuse primary keys with identity descriptors. A primary key is a special value that is guaranteed to be unique within a table. It happens to be used as a logical identity (at least when using surrogate keys). But as NOT NULL constraints, PRIMARY KEY constraints are constraints, and they’re deferrable in some databases.

And there are many ways how we can produce results where primary keys are no longer meaningful, e.g. this:

SELECT * FROM person
UNION ALL
SELECT * FROM person

SQL, unlike relational algebra, doesn’t operate on sets but on bags (or multisets), i.e. data structures that allow for duplicate values. Multisets make analytics much more powerful, while making OLTP quite harder. As always, with useful things, they come at a price.

jOOQ, by consequence, also works in the value-oriented multi set paradigm. This is completely contrary to what Hibernate / JPA does, as Hibernate emulates entity identity through the primary key, which it has to do, being an object-graph persistence API. It doesn’t have to do this because of working with sets rather than multisets, although having identities does make things easier in that paradigm. If you want to read an interesting and entertaining discussion on the subject, check out these tweets between Gavin King and myself:

The importance here is to understand: Neither approach is absolutely better. Both have their advantages. If a RDBMS vendor had implemented a database following a set-based approach instead of SQL’s multiset-based approach, a lot of persistence problems would have been much easier to implement on that RDBMS. On the other hand, a lot of reporting and analytics would have been harder, because with sets being sets, we’d have to constantly prevent “duplicates” from being removed early by keeping primary keys around in queries until the final aggregation.

Now even if we could re-start this interesting discussion, fact is, that we have SQL and it is multiset-based. The default is SELECT "ALL", not SELECT DISTINCT (the ALL keyword being specified in the standard, but not available in most implementations).

When using jOOQ, a value-based record-centric programming approach is recommended, where result sets from jOOQ queries are really “just” streams of records, which will be further transformed without ever thinking about persisting any elements from those streams again. Sure there can be write operations as well, but a jOOQ (or SQL) write operation is also a multiset-based streaming of records (values) back into the database. That’s important to know, because all of

  • INSERT
  • UPDATE
  • DELETE
  • MERGE

statements are multiset-based, i.e. they take a set of values, not just a single row. For instance, INSERT:

-- Not all databases support this standard syntax:
INSERT INTO t (a, b, c)
VALUES (1, 2, 3),
       (4, 5, 6),
       (7, 8, 9);

-- But all databases support this one:
INSERT INTO t1 (a, b, c)
SELECT a, b, c
FROM t2;

Notice how this has absolutely nothing to do with identity-based object-graph persistence. In SQL, we’re always streaming a set of values from one place to another, possibly targeting a table where we store that set. The approach is really beautiful, try to think this way and it’ll open up a whole new world to the SQL-oriented programmer.

In a future article, I’ll even go a step further and claim that SQL is an (almost) completely side-effect free language (and this includes statements like INSERT – stay tuned).

Takeaway

In SQL, everything is a value. There is no identity. It is not needed, because SQL is a multiset-based language, where we’re always operating on the entire data set, not on individual records, even if CRUD operations may make you think otherwise. jOOQ encourages this way of thinking by putting the table and the “value-based” record into the center of the programming model.

3. ResultQuery is an Iterable

I’ve blogged about this before, and some users may have discovered it by accident, intrigued. A jOOQ ResultQuery is an Iterable, meaning that you can “foreach it”:

ResultQuery<?> query =
DSL.using(configuration)
   .select(PERSON.FIRST_NAME, PERSON.LAST_NAME)
   .from(PERSON);

// Java 5 style
for (Record record : query)
  System.out.println(record);

// Java 8 style
query.forEach(System.out::println);

It makes a lot of sense. A SQL query is a description of a set of tuples. SQL is a functional programming language, and if you forget about some concurrency aspects, it is, in principle, side-effect free. This means that the query really IS the set of tuples (another nice way to think about SQL!). With that thought in mind, we can simply iterate it.

To the procedural mind of many Java developers, this might be a bit funky and surprising, but give this a little thought and it might “click”. Consider also this previous article, claiming that streams, for comprehensions, and SQL are all the same:

Or also this fun tweet:

Takeaway

We’re not there yet in Java, we still explicitly iterate, but when we do, and the data source is a SQL query, make it a jOOQ query because that helps you forget about the difference between the query and the data, which are really the same thing in SQL.

4. Ordering is Nice When It’s Cheap. Let’s Retain It

You should avoid ORDER BY in SQL if you don’t really need it. Why? Because unless you can profit from an index that has already pre-ordered your result sets, sorting is a super expensive operation in all programming languages, including SQL. It’s essentially O(n log n).

But let’s assume you do have to sort your results, well, we better want to make sure that this ordering stays the same for as long as possible.

By default, jOOQ returns a Result type, or List types, but there are many utility methods like the ResultQuery.fetchMap() method, which can return something like this:

Map<Integer, String> people =
DSL.using(configuration)
   .select(PERSON.ID, PERSON.FIRST_NAME)
   .from(PERSON)
   .orderBy(PERSON.ID)
   .fetchMap(PERSON.ID, PERSON.FIRST_NAME);

Internally, jOOQ collects all data into a LinkedHashMap, which is a slightly more resource intensive map than the similar HashMap. In case you haven’t used this very often, it’s a map that preserves the insertion order when iterating the map using Map.entrySet() and all the other methods. Quite useful when displaying the map, too. After all, if you do specify the ordering, then you wanted that order to appear in the results, right?

In a similar way, when using Collections.sort() in Java, the sort algorithm guarantees that sorting is stable. If you sort a list twice, then the original ordering will be retained for elements that are not re-ordered. I.e. when sorting by first name, and then by last name, the first name ordering will be retained for equal last names.

Takeaway

ORDER BY is expensive, so if you go through the trouble of actually doing it, you want to retain that order.

5. Dynamic SQL is the Default

In the old days, people mostly wrote static SQL, e.g. using stored procedures in languages like PL/SQL. When you write an implicit cursor loop in PL/SQL:

FOR rec IN (SELECT * FROM person)
LOOP
  dbms_output.put_line(rec.first_name || ' ' || rec.last_name);
END LOOP;

… then, this SQL statement is compiled along with the surrounding procedural code and it will never be changed again. That’s useful for batch processing, reporting, etc. (Strictly speaking it isn’t really “static”, because the SQL statement will still be parsed by the SQL engine like any other query, but the PL/SQL programming model allows for hiding this from you).

In modern days, we require dynamic SQL very often, because the SQL code is often generated from user input. Mostly, because:

  • Users can add predicates through the UI
  • Users can specify aggregations through the UI
  • Users can specify ordering through the UI

In some more remote use-cases, users might also influence the JOIN tree and other parts of a dynamically created query.

From a JDBC perspective, all queries are dynamic, even if you’re doing something like this:

try (ResultSet rs = stmt.executeQuery(
  "SELECT * FROM person"
)) {
  while (rs.next())
    out.println(rs.getString(1) + " " + rs.getString(2));
}

Clearly, the SQL string seems “static” in the way that the Java compiler will compile it once and then never touch it again. The above program will always send the exact same SQL string to the server. Yet from a JDBC API perspective, the string is just an argument to the executeQuery() method, just as if we wrote it like this:

try (ResultSet rs = stmt.executeQuery(
  "SELECT * FROM person" + 
  (active ? " WHERE active = 1" : "")
)) {
  while (rs.next())
    out.println(rs.getString(1) + " " + rs.getString(2));
}

Yuck! String concatenation to build SQL strings. There’s a substantial risk of:

Of course, the above example is SQL injection “safe”, because the SQL string is entirely constructed from constants, not user input. But how quickly could the code be refactored to this?

try (ResultSet rs = stmt.executeQuery(
  "SELECT * FROM person" + 
  (active ? (" WHERE active = " + active) : "")
)) {
  while (rs.next())
    out.println(rs.getString(1) + " " + rs.getString(2));
}

SQL builders like jOOQ help prevent SQL injection, even for dynamic SQL queries. The above query will be written as follows in jOOQ:

for (PersonRecord rec : DSL.using(configuration)
        .selectFrom(person)
		.where(active
		    ? PERSON.ACTIVE.eq(active)
			: trueCondition()))
  out.println(rec.getFirstName() + " " + rec.getLastName());

The active flag check that is added to the SQL query dynamically will default to creating a bind variable, and even if it is inlined, it will be escaped, depending on its type.

The interesting bit here, however, is that the jOOQ query is always a dynamic SQL query. The above approach used an inline expression to decide whether a certain predicate needs to be added to the statement. If that predicate gets more complex, we can extract the construction of the predicate to a local variable, or a function.

Local variable

Condition condition = trueCondition();

if (active)
  condition = PERSON.ACTIVE.eq(active);
	
if (searchForFirstName)
  condition = condition.and(PERSON.FIRST_NAME.like(pattern));

for (PersonRecord rec : DSL.using(configuration)
        .selectFrom(person)
		.where(condition))
  out.println(rec.getFirstName() + " " + rec.getLastName());

This is quite neat.

Functions

Or, if things get even more complex, we might like to factor out the logic to a method, or a function. Some people have started calling such an approach “functional relational mapping”:

Condition personCondition(boolean active, String pattern) {
  Condition condition = trueCondition();

  if (active)
    condition = PERSON.ACTIVE.eq(active);
	
  if (pattern != null)
    condition = condition.and(PERSON.FIRST_NAME.like(pattern));
	
  return condition;
}

// And then:
for (PersonRecord rec : DSL.using(configuration)
        .selectFrom(person)
		.where(personCondition(active, pattern)))
  out.println(rec.getFirstName() + " " + rec.getLastName());

Or even:

BiFunction<Boolean, String, Condition> personCondition() {
  return (active, pattern) -> {
    Condition condition = trueCondition();

    if (active)
      condition = PERSON.ACTIVE.eq(active);
	
    if (pattern != null)
      condition = condition.and(PERSON.FIRST_NAME.like(pattern));
	
    return condition;
  };
}

// And then:
for (PersonRecord rec : DSL.using(configuration)
        .selectFrom(person)
		.where(personCondition.apply(active, pattern)))
  out.println(rec.getFirstName() + " " + rec.getLastName());

Not only is this approach to writing dynamic SQL extremely useful for client code that relies on dynamic SQL, the expression tree that is built behind the scenes is also available at runtime for more complex transformations, such as applying row level security to certain queries, or more simply to apply something like schema-based multi-tenancy. While the Java code stays exactly the same, the generated SQL string may be transformed by your own library code, behind the scenes.

Static SQL

Of course, jOOQ doesn’t imply that you have to write dynamic SQL. You can store jOOQ-generated SQL strings in caches, or you can use stored procedures with jOOQ. In fact, jOOQ encourages you to use stored procedures!

Takeaway

Dynamic SQL is really useful. jOOQ defaults to writing dynamic SQL in a way that you don’t even notice. A SQL query is a function just as much as it is a collection description. jOOQ helps you think about SQL this way.

Conclusion

SQL is a beautiful language with an interesting syntax. If we look at the concepts that are the foundation of the SQL language, we see that SQL queries are functional / declarative collection descriptions. With this paradigm in mind, we can write really powerful SQL statements, and jOOQ encourages this as this paradigm is at the core of the jOOQ API design.

Enjoy writing functional-relational mapping code.

10 Features I Wish Java Would Steal From the Kotlin Language

This article is overdue. After the hype around the release of Kotlin 1.0 has settled, let’s have a serious look at some Kotlin language features that we should have in Java as well.

In this article, I’m not going to wish for unicorns. But there are some low hanging fruit (as far as I naively can see), which could be introduced into the Java language without great risk. While you’re reading this article, be sure to copy paste examples to http://try.kotlinlang.org, an online REPL for Kotlin

1. Data class

Language designers hardly ever agree on the necessity and the feature scope of what a class is. In Java, curiously, every class always has identity a concept that is not really needed in 80% – 90% of all real world Java classes. Likewise, a Java class always has a monitor on which you can synchronize.

In most cases, when you write a class, you really just want to group values, like Strings, ints, doubles. For instance:

public class Person {
    final String firstName;
    final String lastName;
    public JavaPerson(...) {
        ...
    }
    // Getters
    ...

    // Hashcode / equals
    ...

    // Tostring
    ...

    // Egh...
}

By the time you’ve finished typing all of the above, your fingers will no longer be. Java developers have implemented ugly workarounds for the above, like IDE code generation, or lombok, which is the biggest of all hacks. In a better Java, nothing in Lombok would really be needed.

As, for instance, if Java had Kotlin’s data classes:

data class Person(
  val firstName: String,
  val lastName: String
)

The above is all we need to declare the equivalent of the previous Java code. Because a data class is used to store data (duh), i.e. values, the implementation of things like hashCode(), equals(), toString() is obvious and can be provided by default. Furthermore, data classes are first class tuples, so they can be used as such, e.g. to destructure them again in individual references:

val jon = Person("Jon", "Doe") 
val (firstName, lastName) = jon

In this case, we may hope. Valhalla / Java 10 is being designed and with it, value types. We’ll see how many features will be provided on the JVM directly, and in the Java language. This will certainly be an exciting addition.

Notice how val is possible in Kotlin: Local variable type inference. This is being discussed for a future Java version right now.

2. Defaulted parameters

How many times do you overload an API like the following:

interface Stream<T> {
    Stream<T> sorted();
    Stream<T> sorted(Comparator<? super T> comparator);
}

The above are exactly the same JDK Stream operations. The first one simply applies Comparator.naturalOrder() to the second one. So we could write the following, in Kotlin:

fun sorted(comparator : Comparator<T> 
         = Comparator.naturalOrder()) : Stream<T>

The advantage of this isn’t immediately visible, when there is only one defaulted parameter. But imagine a function with tons of optional parameters:

fun reformat(str: String,
             normalizeCase: Boolean = true,
             upperCaseFirstLetter: Boolean = true,
             divideByCamelHumps: Boolean = false,
             wordSeparator: Char = ' ') {
...
}

Which can be called in any of the following ways:

reformat(str)
reformat(str, true, true, false, '_')
reformat(str,
  normalizeCase = true,
  upperCaseFirstLetter = true,
  divideByCamelHumps = false,
  wordSeparator = '_'
)

The power of defaulted parameters is that they are especially useful when passing arguments by name, rather than by index. This is currently not supported in the JVM, which until Java 8, doesn’t retain the parameter name at all (in Java 8, you can turn on a JVM flag for this, but with all of Java’s legacy, you shouldn’t rely on this yet).

Heck, this feature is something I’m using in PL/SQL every day. Of course, in Java, you can work around this limitation by passing a parameter object.

3. Simplified instanceof checks

If you will, this is really an instanceof switch. Some people may claim that this stuff is evil, bad OO design. Nja nja. I say, this happens every now and then. And apparently, in Java 7, string switches were considered sufficiently common to modify the language to allow them. Why not instanceof switches?

val hasPrefix = when(x) {
  is String -> x.startsWith("prefix")
  else -> false
}

Not only is this doing an instanceof switch, it is doing it in the form of an assignable expression. Kotlin’s version of this when expression is powerful. You can mix any sort of predicate expressions, similar to SQL’s CASE expression. For instance, this is possible as well:

when (x) {
  in 1..10 -> print("x is in the range")
  in validNumbers -> print("x is valid")
  !in 10..20 -> print("x is outside the range")
  else -> print("none of the above")
}

Compare to SQL (not implemented in all dialects):

CASE x
  WHEN BETWEEN 1 AND 10 THEN 'x is in the range'
  WHEN IN (SELECT * FROM validNumbers) THEN 'x is valid'
  WHEN NOT BETWEEN 10 AND 20 'x is outside the range'
  ELSE 'none of the above'
END

As you can see, only SQL is more powerful than Kotlin.

4. Map key / value traversal

Now this could really be done very easily only with syntax sugar. Granted, having local variable type inference would already be a plus, but check this out

val map: Map<String, Int> = ...

And now, you can do:

for ((k, v) in map) {
    ...
}

After all, most of the time when traversing a map, it’ll be by Map.entrySet(). Map could have been enhanced to extend Iterable<Entry<K, V>> in Java 5, but hasn’t. That’s really a pity. After all, it has been enhanced in Java 8 to allow for internal iteration over the entry set in Java 8 via Map.forEach():

map.forEach((k, v) -> {
    ...
});

It’s not too late, JDK gods. You can still let Map<K, V> extend Iterable<Entry<K, V>>

5. Map access literals

This one is something that would add tons and tons of value to the Java language. We have arrays, like most other languages. And like most other languages, we can access array elements by using square brackets:

int[] array = { 1, 2, 3 };
int value = array[0];

Note also the fact that we have array initialiser literals in Java, which is great. So, why not also allow for accessing map elements with the same syntax?

val map = hashMapOf<String, Int>()
map.put("a", 1)
println(map["a"])

In fact, x[y] is just syntax sugar for a method call backed by x.get(y). This is so great, we have immediately proceeded with renaming our Record.getValue() methods in jOOQ to Record.get() (leaving the old ones as synonyms, of course), such that you can now dereference your database record values as such, in Kotlin

ctx.select(a.FIRST_NAME, a.LAST_NAME, b.TITLE)
   .from(a)
   .join(b).on(a.ID.eq(b.AUTHOR_ID))
   .orderBy(1, 2, 3)
   .forEach {
       println("""${it[b.TITLE]} 
               by ${it[a.FIRST_NAME]} ${it[a.LAST_NAME]}""")
   }

Since jOOQ holds all column type information on individual record columns, you can actually know in advance that it[b.TITLE] is a String expression. Great, huh? So, not only can this syntax be used with JDK maps, it can be used with any library that exposes the basic get() and set() methods.

Stay tuned for more jOOQ and Kotlin examples here:
https://github.com/jOOQ/jOOQ/blob/master/jOOQ-examples/jOOQ-kotlin-example/src/main/kotlin/org/jooq/example/kotlin/FunWithKotlinAndJOOQ.kt

6. Extension functions

This one is a controversial topic, and I can perfectly understand when language designers stay clear of it. But every now and then, extension functions are very useful. The Kotlin syntax here is actually just for a function to pretend to be part of the receiver type:

fun MutableList<Int>.swap(index1: Int, index2: Int) {
  val tmp = this[index1] // 'this' corresponds to the list
  this[index1] = this[index2]
  this[index2] = tmp
}

This will now allow for swapping elements in a list:

val l = mutableListOf(1, 2, 3)
l.swap(0, 2)

This would be very useful for libraries like jOOλ, which extends the Java 8 Stream API by wrapping it in a jOOλ type (another such library is StreamEx, with a slightly different focus). The jOOλ Seq wrapper type is not really important, as it pretends to be a Stream on steroids. It would be great, if jOOλ methods could be put onto Stream artificially, just by importing them:

list.stream()
    .zipWithIndex()
    .forEach(System.out::println);

The zipWithIndex() method isn’t really there. The above would just translate to the following, less readable code:

seq(list.stream())
    .zipWithIndex()
    .forEach(System.out::println);

In fact, extension methods would even allow to bypass wrapping everything explicitly in a stream(). For instance, you could then do:

list.zipWithIndex()
    .forEach(System.out::println);

As all of jOOλ’s method could be designed to also be applied to Iterable.

Again, this is a controversial topic. For instance, because

While giving the illusion of being virtual, extension functions really are just sugared static methods. It’s a significant risk for object oriented application design to engage in that trickery, which is why this feature probably won’t make it into Java.

7. Safe-call operator (and also: Elvis operator)

Optional is meh. It’s understandable that an Optional type needed to be introduced in order to abstract over the absence of primitive type values, which cannot be null. We now have things like OptionalInt, e.g. to model things like:

OptionalInt result =
IntStream.of(1, 2, 3)
         .filter(i -> i > 3)
         .findFirst();

// Agressive programming ahead
result.orElse(OR_ELSE);

Optional is a monad

Yes. It allows you to flatMap() the absent value.

o_O

Sure, if you want to do sophisticated functional programming, you’ll start typing map() and flatMap() everywhere. Like today, when we’re typing getters and setters. Along will come lombok generating flatmapping calls, and Spring will add some @AliasFor style annotation for flatmapping. And only the enlightened will be able to decipher your code.

When all we needed was just a simple null safety operator before getting back to daily business. Like:

String name = bob?.department?.head?.name

I really like this type of pragmatism in Kotlin. Or do you prefer (flat)mapping?

Optional<String> name = bob
    .flatMap(Person::getDepartment)
    .map(Department::getHead)
    .flatMap(Person::getName);

Can you read this? I cannot. Neither can I write this. If you get this wrong, you’ll get boxoxed.

Of course, Ceylon is the only language that got nulls right. But Ceylon has tons of features that Java will not get before version 42, and I’m not wishing for unicorns. I’m wishing for the safe-call operator (and also the elvis operator, which is slightly different), which could be implemented in Java too. The above expression is just syntax sugar for:

String name = null;
if (bob != null) {
    Department d = bob.department
    if (d != null) {
        Person h = d.head;
        if (h != null)
            name = h.name;
    }
}

What can possibly be wrong with that simplification?

8. Everything is an expression

Now this might just be a unicorn. I don’t know if there is a JLS / parser limitation that will forever keep us in the misery of prehistoric distinction between statement and expression.

At some point in time, people have started using statements for things that yield side-effects, and expressions for more functional-ish things. It is thus not surprising, that all String methods are really expressions, operating on an immutable string, returning a new string all the time.

This doesn’t seem to go well with, for instance, if-else in Java, which is expected to contain blocks and statements, each possibly yielding side-effects.

But is that really a requirement? Can’t we write something like this in Java as well?

val max = if (a > b) a else b

OK, we have this weird conditional expression using ?:. But what about Kotlin’s when (i.e. Java’s switch)?

val hasPrefix = when(x) {
  is String -> x.startsWith("prefix")
  else -> false
}

Isn’t that much more useful than the following equivalent?

boolean hasPrefix;

if (x instanceof String)
    hasPrefix = x.startsWith("prefix");
else
    hasPrefix = false;

(yes, I know about ?:. I just find if-else easier to read, and I don’t see why that should be a statement, not an expression. Heck, in Kotlin, even try is an expression, not a statement:

val result = try {
    count()
} catch (e: ArithmeticException) {
    throw IllegalStateException(e)
}

Beautiful!

9. Single expression functions

Now this. This would save so much time reading and writing simple glue code. And in fact, we already have the syntax in annotations. Check out Spring’s magical @AliasFor annotation, for instance. It yields:

public @interface AliasFor {
    @AliasFor("attribute")
    String value() default "";
    @AliasFor("value")
    String attribute() default "";
}

Now, if you squint really hard, these are just methods yielding constant values, because annotations are just interfaces with generated byte code for their implementations. We can discuss syntax. Of course, this irregular usage of default is weird, given that it was not re-used in Java 8 for default methods, but I guess Java always needs the extra syntax so developers feel alive as they can better feel their typing fingers. That’s OK. We can live with that. But then again, why do we have to? Why not just converge to the following?

public @interface AliasFor {
    String value() = "";
    String attribute() = "";
}

And the same also for class / interface default methods?

// Stop pretending this isn't an interface
public interface AliasFor {
    String value() = "";
    String attribute() = "";
}

Now that would look nice. But given Java’s existing syntax, this might just be a unicorn, so let’s move on to…

10. Flow-sensitive typing

Now this. THIS!

We’ve blogged about sum types before. Java has sum types with exceptions since Java 7:

try {
    ...
}
catch (IOException | SQLException e) {
    // e can be of type IOException and/or SQLException
    // within this scope
}

But Java, unfortunately, doesn’t have flow-sensitive typing. Flow-sensitive typing is of the essence in a language that supports sum types, but it is also useful otherwise. For instance, in Kotlin:

when (x) {
    is String -> println(x.length)
}

We don’t need to cast, obviously, because we already checked that x is String. Conversely, in Java:

if (x instanceof String)
    System.out.println(((String) x).length());

Aaagh, all this typing. IDE autocompletion is smart enough to offer a contextual type’s methods already and then generate the unnecessary cast for you. But it would be great if this was never needed, every time we explicitly narrow a type using control flow structures.

For more info, see this wikipedia entry about flow sensitive typing. A feature that could absolutely be added to the Java language. After all, we already got flow-sensitive final local variables since Java 8.

11. (Bonus) Declaration site variance

Last but not least, better generics via declaration site variance. Many other languages know this, for instance also C#’s IEnumerable:

public interface IEnumerable<out T> : IEnumerable

The keyword out here means that the generic type T is produced from the type IEnumerable (as opposed to in, which stands for consumption). In C#, Scala, Ceylon, Kotlin, and many other languages, we can declare this on the type declaration, rather than on its usage (although, many languages allow for both). In this case, we say that IEnumerable is covariant with its type T, which means again that IEnumerable<Integer> is a subtype of IEnumerable<Object>

In Java, this isn’t possible, which is why we have a bazillion question by Java newbies on Stack Overflow. Why can’t I…

Iterable<String> strings = Arrays.asList("abc");
Iterable<Object> objects = strings; // boom

In languages like Kotlin, the above would be possible. After all, why shouldn’t it? A thing that can produce strings can also produce objects, and we can even use it in this way in Java:

Iterable<String> strings = Arrays.asList("abc");
for (Object o : strings) {
    // Works!
}

The lack of declaration site variance has made a lot of APIs very intelligible. Consider Stream:

<R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper);

This is just noise. A function is contravariant with its argument type and covariant with its result type by nature a better definition of Function or Stream would be:

interface Function<in T, out R> {}
interface Stream<out T> {}

If this were possible, all that ? super and ? extends garbage could be removed without losing any functionality.

In case you’re wondering what I’m even talking about? 🙂

The great news is, this is being discussed for a (near) future version of Java:
http://openjdk.java.net/jeps/8043488

Conclusion

Kotlin is a promising language, even if it is very late to a game that already seems to have been decided, not in favour of alternative languages on the JVM. Nonetheless, it is a very interesting language to learn from, and with a lot of very good decisions made about some simple things.

Some of these decisions will hopefully be picked up by the Java language gods and integrated into Java. This list here shows some features that might be “easy” to add.

More info about Kotlin idioms:
https://kotlinlang.org/docs/reference/idioms.html

Liked this article?

Read on here:

Java 8 Friday: Language Design is Subtle

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Language Design is Subtle

It’s been a busy week for us. We have just migrated the jOOQ integration tests to Java 8 for two reasons:

  • We want to be sure that client code compiles with Java 8
  • We started to get bored of writing the same old loops over and over again

The trigger was a loop where we needed to transform a SQLDialect[] into another SQLDialect[] calling .family() on each array element. Consider:

Java 7

SQLDialect[] families = 
    new SQLDialect[dialects.length];
for (int i = 0; i < families.length; i++)
    families[i] = dialects[i].family();

Java 8

SQLDialect[] families = 
Stream.of(dialects)
      .map(d -> d.family())
      .toArray(SQLDialect[]::new);

OK, it turns out that the two solutions are equally verbose, even if the latter feels a bit more elegant. 🙂

And this gets us straight into the next topic:

Backwards-compatibility

For backwards-compatibility reasons, arrays and the pre-existing Collections API have not been retrofitted to accommodate all the useful methods that Streams now have. In other words, an array doesn’t have a map() method, just as much as List doesn’t have such a method. Streams and Collections/arrays are orthogonal worlds. We can transform them into each other, but they don’t have a unified API.

This is fine in everyday work. We’ll get used to the Streams API and we’ll love it, no doubt. But because of Java being extremely serious about backwards compatibility, we will have to think about one or two things more deeply.

Recently, we have published a post about The Dark Side of Java 8. It was a bit of a rant, although a mild one in our opinion (and it was about time to place some criticism, after all the praise we’ve been giving Java 8 in our series, before 😉 ). First off, that post triggered a reaction by Edwin Dalorzo from our friends at Informatech. (Edwin has written this awesome post comparing LINQ and Java 8 Streams, before). The criticism in our article evolved around three main aspects:

  • Overloading getting more complicated (see also this compiler bug)
  • Limited support for method modifiers on default methods
  • Primitive type “API overloads” for streams and functional interfaces

A response by Brian Goetz

I then got a personal mail from no one less than Brian Goetz himself (!), who pointed out a couple of things to me that I had not yet thought about in this way:

I still think you’re focusing on the wrong thing. Its not really the syntax you don’t like; its the model — you don’t want “default methods”, you want traits, and the syntax is merely a reminder that you didn’t get the feature you wanted. (But you’d be even more confused about “why can’t they be final” if we dropped the “default” keyword!) But that’s blaming the messenger (where here, the keyword is the messenger.)

Its fair to say “this isn’t the model I wanted”. There were many possible paths in the forest, and it may well be the road not taken was equally good or better.

This is also what Edwin had concluded. Default methods were a necessary means to tackle all the new API needed to make Java 8 useful. If Iterator, Iterable, List, Collection, and all the other pre-existing interfaces had to be adapted to accommodate lambdas and Streams API interaction, the expert group would have needed to break an incredible amount of API. Conversely, without adding these additional utility methods (see the awesome new Map methods, for instance!), Java 8 would have been only half as good.

And that’s it.

Even if maybe, some more class building tools might have been useful, they were not in the center of focus for the expert group who already had a lot to do to get things right. The center of focus was to provide a means for API evolution. Or in Brian Goetz’s own words:

Reaching out to the community

It’s great that Brian Goetz reaches out to the community to help us get the right picture about Java 8. Instead of explaining rationales about expert group decisions in private messages, he then asked me to publicly re-ask my questions again on Stack Overflow (or lambda-dev), such that he can then publicly answer them. For increased publicity and greater community benefit, I chose Stack Overflow. Here are:

The amount of traction these two questions got in no time shows how important these things are to the community, so don’t miss reading through them!

“Uncool”? Maybe. But very stable!

Java may not have the “cool” aura that node.js has. You may think about JavaScript-the-language whatever you want (as long as it contains swear words), but from a platform marketing perspective, Java is being challenged for the first time in a long time – and being “uncool” and backwards-compatible doesn’t help keeping developers interested.

But let’s think long-term, instead of going with trends. Having such a great professional platform like the Java language, the JVM, the JDK, JEE, and much more, is invaluable. Because at the end of the day, the “uncool” backwards-compatibility can also be awesome. As mentioned initially, we have upgraded our integration tests to Java 8. Not a single compilation error, not a single bug. Using Eclipse’s BETA support for Java 8, I could easily transform anonymous classes into lambdas and write awesome things like these upcoming jOOQ 3.4 nested transactions (API not final yet):

ctx.transaction(c1 -> {
    DSL.using(c1)
       .insertInto(AUTHOR, AUTHOR.ID, AUTHOR.LAST_NAME)
       .values(3, "Doe")
       .execute();

    // Implicit savepoint here
    try {
        DSL.using(c1).transaction(c2 -> {
            DSL.using(c2)
               .update(AUTHOR)
               .set(AUTHOR.FIRST_NAME, "John")
               .where(AUTHOR.ID.eq(3))
               .execute();

            // Rollback to savepoint
            throw new MyRuntimeException("No");
        });
    }

    catch (MyRuntimeException ignore) {}

    return 42;
});

So at the end of the day, Java is great. Java 8 is a tremendous improvement over previous versions, and with great people in the expert groups (and reaching out to the community on social media), I trust that Java 9 will be even better. In particular, I’m looking forward to learning about how these two projects evolve:

Although, again, I am really curious how they will pull these two improvements off from a backwards-compatibility perspective, and what caveats we’ll have to understand, afterwards. 😉

Anyway, let’s hope the expert groups will continue to provide public feedback on Stack Overflow. Stay tuned for more awesome Java 8 content on this blog.