How to Ensure Your Code Works With Older JDKs

jOOQ is a very backwards compatible product. This doesn’t only mean that we keep our own API backwards compatible as well as possible, but we also still support Java 6 in our commercial distributions.

In a previous blog post, I’ve shown how we manage to support Java 6 while at the same time not missing out on cool Java 8 language and API features, such as Stream and Optional support. For instance, you can do this with jOOQ’s ordinary distribution:

// Fetching 0 or 1 actors
Optional<Record2<String, String>> actor =, ACTOR.LAST_NAME)

// Fetching a stream of actors
try (Stream<Record2<String, String>> actor = ctx
       .fetchStream()) {

This API is present in jOOQ’s ordinary distribution and it is stripped from that distribution prior to building the Java 6 distribution.

But what about the JDK’s more subtle APIs?

It is relatively easy to remember not to use Streams, Optionals, lambdas, method references, default methods lightheartedly in your library’s code. After all, those were all major changes to Java 8 and we can easily add our API removal markers around those parts. And even if we forgot, building the Java 6 distribution would quite probably fail, because Streams are very often used with lambdas, in case of which a compiler that is configured for Java version 1.6 will not compile the code.

But recently, we’ve had a more subtle bug, #6860. jOOQ API was calling java.lang.reflect.Method.getParameterCount(). Since we compile jOOQ’s Java 6 distribution with Java 8, this didn’t fail. The sources were kept Java 6 language compatible, but not JDK 6 API compatible, and unfortunately, there’s no option in javac, nor in the Maven compiler plugin to do such a check.

Why not use Java 6 to compile the Java 6 distribution?

The reason why we’re using Java 8 to build jOOQ’s Java 6 distribution is the fact that Java 8 “fixed” a lot (and I mean a lot) of very old and weird edge cases related to generics,
overloading, varargs, and all that stuff. While this might be irrelevant for ordinary APIs, for jOOQ it is not. We really push the limits of what’s possible with the Java language.

So, we’re paying a price for building jOOQ’s Java 6 distribution with Java 8. We’re flying in “stealth mode”, not 100% sure whether our JDK API usage is compliant.

Luckily, the JDK doesn’t change much between releases, so a lot of stuff from JDK 8 was already there in JDK 6. Also, our integration tests would fail, if we did accidentally use a method like the above. Unfortunately, that particular method call simply slipped by the integration tests (there will never be enough tests for every scenario).

The solution

Apart from fixing the trivial bug and avoiding that particular method, we’ve now added the cool “animal sniffer” Maven plugin to our Java 6 build, whose usage you can see here:

All we needed to add to our Java 6 distribution profile was this little snippet:


This will then produce a validation error like the following:

[INFO] --- animal-sniffer-maven-plugin:1.16:check (default) @ jooq-codegen ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java16:1.0
[ERROR] C:\..\ Undefined reference: int java.lang.reflect.Method.getParameterCount()
[ERROR] C:\..\ Undefined reference: int java.lang.reflect.Method.getParameterCount()


What the sun.misc.Unsafe Misery Teaches Us

Oracle will remove the internal sun.misc.Unsafe class in Java 9. While most people are probably rather indifferent regarding this change, some other people – mostly library developers – are not. There had been a couple of recent articles in the blogosphere painting a dark picture of what this change will imply:

Maintaining a public API is extremely difficult, especially when the API is as popular as that of the JDK. There is simply (almost) no way to keep people from shooting themselves in the foot. Oracle (and previously Sun) have always declared the sun.* packages as internal and not to be used. Citing from the page called “Why Developers Should Not Write Programs That Call ‘sun’ Packages”:

The sun.* packages are not part of the supported, public interface.

A Java program that directly calls into sun.* packages is not guaranteed to work on all Java-compatible platforms. In fact, such a program is not guaranteed to work even in future versions on the same platform.

This disclaimer is just one out of many similar disclaimers and warnings. Whoever goes ahead and uses Unsafe does so … “unsafely“.

What do we learn from this?

The concrete solution to solving this misery is being discussed and still open. A good idea would be to provide a formal and public replacement before removing Unsafe, in order to allow for migration paths of the offending libraries.

But there’s a more important message to all of this. The message is:

When all you have is a hammer, every problem looks like a thumb

Translated to this situation: The hammer is Unsafe and given that it’s a very poor hammer, but the only option, well, library developers might just not have had much of a choice. They’re not really to blame. In fact, they took a gamble in one of the world’s most stable and backwards compatible software environments (= Java) and they fared extremely well for more than 10 years. Would you have made a different choice in a similar situation? Or, let me ask differently. Was betting on AWT or Swing a much safer choice at the time?

If something can somehow be used by someone, then it will be, no matter how obviously they’re gonna shoot themselves in the foot. The only way to currently write a library / API and really prevent users from accessing internals is to put everything in a single package and make everything package-private. This is what we’ve been doing in jOOQ from the beginning, knowing that jOOQ’s internals are extremely delicate and subject to change all the time.

For more details about this rationale, read also:

However, this solution has a severe drawback for those developing those internals. It’s a hell of a package with almost no structure. That makes development rather difficult.

What would be a better Java, then?

Java has always had an insufficient set of visibilities:

  • public
  • protected
  • default (package-private)
  • private

There should be a fifth visibility that behaves like public but prevents access from “outside” of a module. In a way, that’s between the existing public and default visibilities. Let’s call this the hypothetical module visibility.

In fact, not only should we be able to declare this visibility on a class or member, we should be able to govern module inter-dependencies on a top level, just like the Ceylon language allows us to do:

module org.hibernate "3.0.0.beta" {
    import ceylon.collection "1.0.0";
    import java.base "7";
    shared import java.jdbc "7";

This reads very similar to OSGi’s bundle system, where bundles can be imported / exported, although the above module syntax is much much simpler than configuring OSGi.

A sophisticated module system would go even further. Not only would it match OSGi’s features, it would also match those of Maven. With the possibility of declaring dependencies on a Java language module basis, we might no longer need the XML-based Maven descriptors, as those could be generated from a simple module syntax (or Gradle, or ant/ivy).

And with all of this in place, classes like sun.misc.Unsafe could be declared as module-visible for only a few JDK modules – not the whole world. I’m sure the number of people abusing reflection to get a hold of those internals would decrease by 50%.


I do hope that in a future Java, this Ceylon language feature (and also Fantom language feature, btw) will be incorporated into the Java language. A nice overview of Java 9 / Jigsaw’s modular encapsulation can be seen in this blog post:

The Features Project Jigsaw Brings To Java 9

Until then, if you’re an API designer, do know that all disclaimers won’t work. Your internal APIs will be used and abused by your clients. They’re part of your ordinary public API from day 1 after you publish them. It’s not your user’s fault. That’s how things work.

Don’t be Fooled by Generics and Backwards-Compatibility. Use Generic Generic Types

I’ve recently had a very interesting discussion with Sebastian Gruber from Ergon, a very early jOOQ customer, whom we’re in close touch with. Talking to Sebastian has lead our engineering team to the conclusion that we should completely rewrite the jOOQ API. Right now, we already have lots of generics for various purposes, e.g.

  • Generics for column types, such as
    interface Field<T> { ... }
    Field<String> field = BOOK.TITLE;
  • Generics for table types, such as
    interface Table<R extends Record> { ... }
    Table<BookRecord> books = BOOK;
  • Combined generics where both <T> and <R> are used
  • … and much more

Sometimes, you just cannot anticipate how many different generic types you’ll need on your classes and interfaces two years down the line, and the problem with Java is: You can generify your classes only exactly once. Let’s assume that you’ve always had a type like this:

class Foo {}

Now you happen to know that you need two generic type parameters right now:

// Still compatible
class Foo<Bar, Baz> {}

That’ll work and all the existing client code will still compile, with a rawtype warning. But once you’ve published Foo<Bar, Baz>, you can no longer add more type variables to it, or remove them. Every modification will break client code!

// Breaking change
class Foo<Bar, Baz, Fizz> {}

The solution: Generic generic types

We don’t want to place that burden upon our customers, the heavy burden of backwards-incompatibility. This is why we’re now publishing our next release of jOOQ with a new feature that we call generic generic types. How does it work? It’s easy. We’ve learned from the best database designers who have already been using generic column types all along. In SQL, if you run into this kind of problem, you’d simply write:

    bar int,
    baz int,
    fizz int,

    generic_1 varchar(4000),
    generic_2 varchar(4000),
    generic_3 varchar(4000),
    generic_4 varchar(4000),
    -- [...]

Now your SQL schema is safe for ages to come. We’ll do the same in Java:

class Foo<

    // [...]
> {}

We’ll thus generify all our types to have exactly 256 generic type parameters. 256 was the sensible limit that MS Access chose for the number of possible columns. That way, our customers will only have to upgrade to the new version of jOOQ once and from then on, generic type backwards-compatibility will be guaranteed forever.

Happy coding!

The Java Legacy is Constantly Growing

I’ve recently stumbled upon a very interesting caveat of the JDK APIs, the Class.getConstructors() method. Its method signature is this:

Constructor<?>[] getConstructors()

The interesting thing here is that Class.getConstructor(Class...) returns a Constructor<T>, with <T> being maintained:

Constructor<T> getConstructor(Class<?>... parameterTypes)

Why is there a difference, i.e. why doesn’t the first method return Constructor<T>[]?

Let’s consider the Javadoc:

Note that while this method returns an array of Constructor<T> objects (that is an array of constructors from this class), the return type of this method is Constructor<?>[] and not Constructor<T>[] as might be expected. This less informative return type is necessary since after being returned from this method, the array could be modified to hold Constructor objects for different classes, which would violate the type guarantees of Constructor<T>[].


That’s a tough one. Historically, here’s how this happened:

Java 1.0 / Oak: Arrays

In Java 1.0 (the immediate successor of the Oak programming language), arrays were already introduced. In fact, they have been introduced before the collections API, which was introduced in Java 1.2. Arrays suffer from all the problems that we know today, including them being covariant, which leads to a lot of problems at runtime, that cannot be checked at compile time:

Object[] objects = new String[1];
objects[0] = Integer.valueOf(1); // Ouch

Java 1.1: Reflection API

Short of a “decent” collections API, the only possible return type of the Class.getConstructors() method was Constructor[]. A reasonable decision at the time. Of course, you could do the same mistake as above:

Object[] objects = String.class.getConstructors();
objects[0] = Integer.valueOf(1); // Ouch

but in the addition to the above, you could also, rightfully, write this:

Constructor[] constructors  = String.class.getConstructors();
constructors[0] = Object.class.getConstructor();

// Muahahahahahahaha

Java 1.2: Collections API

Java has been backwards-compatible from the very early days, even from Oak onwards. There’s a very interesting piece of historic research about some of Oak’s backwards-compatibility having leaked into Java to this date in this Stack Overflow question.

While it would have been natural to design the reflection API using collections, now, it was already too late. A better solution might’ve been:

List getConstructors()

However, note that we didn’t have generics yet, so the array actually conveys more type information than the collection.

Java 1.5: Generics

In Java 5, the change from

Constructor[] getConstructors()


Constructor<?>[] getConstructors()

has been made for the reasons mentioned above. Now, the alternative API using a collection would definitely have been better:

List<Constructor<T>> getConstructors()

But the ship has sailed.

Java, the ugly wart

Java is full of these little caveats. They’re all documented in the Javadocs, and often on Stack Overflow. Just yesterday, we’ve documented a new caveat related to completely new API in Map and ConcurrentHashMap.

“Stewardship: the Sobering Parts,” a very good talk about all those caveats and how hard it is to maintain them by Brian Goetz can be seen here:

The summary of the talk:

When language designers talk about the language they're designing

When language designers talk about the language they’re designing

How Nashorn Impacts API Evolution on a New Level

Following our previous article about how to use jOOQ with Java 8 and Nashorn, one of our users discovered a flaw in using the jOOQ API as discussed here on the user group. In essence, the flaw can be summarised like so:

Java code

package org.jooq.nashorn.test;

public class API {
    public static void test(String string) {
        throw new RuntimeException("Don't call this");

    public static void test(Integer... args) {

JavaScript code

var API = Java.type("org.jooq.nashorn.test.API");
API.test(1); // This will fail with RuntimeException

After some investigation and the kind help of Attila Szegedi, as well as Jim Laskey (both Nashorn developers from Oracle), it became clear that Nashorn disambiguates overloaded methods and varargs differently than what an old Java developer might expect. Quoting Attila:

Nashorn’s overload method resolution mimics Java Language Specification (JLS) as much as possible, but allows for JavaScript-specific conversions too. JLS says that when selecting a method to invoke for an overloaded name, variable arity methods can be considered for invocation only when there is no applicable fixed arity method.

I agree that variable arity methods can be considered only when there is no applicable fixed arity method. But the whole notion of “applicable” itself is completely changed as type promotion (or coercion / conversion) using ToString, ToNumber, ToBoolean is preferred over what intuitively appear to be “exact” matches with varargs methods!

Let this sink in!

Given that we now know how Nashorn resolves overloading, we can see that any of the following are valid workarounds:

Explicitly calling the test(Integer[]) method using an array argument:

This is the simplest approach, where you ignore the fact that varargs exist and simply create an explicit array

var API = Java.type("org.jooq.nashorn.test.API");

Explicitly calling the test(Integer[]) method by saying so:

This is certainly the safest approach, as you’re removing all ambiguity from the method call

var API = Java.type("org.jooq.nashorn.test.API");

Removing the overload:

public class AlternativeAPI1 {
    public static void test(Integer... args) {

Removing the varargs:

public class AlternativeAPI3 {
    public static void test(String string) {
        throw new RuntimeException("Don't call this");

    public static void test(Integer args) {

Providing an exact option:

public class AlternativeAPI4 {
    public static void test(String string) {
        throw new RuntimeException("Don't call this");

    public static void test(Integer args) {
        test(new Integer[] { args });

    public static void test(Integer... args) {

Replacing String by CharSequence (or any other “similar type”):

Now, this is interesting:

public class AlternativeAPI5 {
    public static void test(CharSequence string) {
        throw new RuntimeException("Don't call this");

    public static void test(Integer args) {

Specifically, the distinction between CharSequence and String types appears to be very random from a Java perspective in my opinion.

Agreed, implementing overloaded method resolution in a dynamically typed language is very hard, if even possible. Any solution is a compromise that will introduce flaws at some ends. Or as Attila put it:

As you can see, no matter what we do, something else would suffer; overloaded method selection is in a tight spot between Java and JS type systems and very sensitive to even small changes in the logic.

True! But not only is overload method selection very sensitive to even small changes. Using Nashorn with Java interoperability is, too! As an API designer, over the years, I have grown used to semantic versioning, and the many subtle rules to follow when keeping an API source compatible, behavior compatible – and if ever possible – to a large degree also binary compatible.

Forget about that when your clients are using Nashorn. They’re on their own. A newly introduced overload in your Java API can break your Nashorn clients quite badly. But then again, that’s JavaScript, the language that tells you at runtime that:


… yields

[10, NaN, 2, 3]

… and where

++[[]][+[]]+[+[]] === "10"

yields true! (sources here)

For more information about JavaScript, please visit this introductory tutorial

Java 8 Friday: Language Design is Subtle

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Language Design is Subtle

It’s been a busy week for us. We have just migrated the jOOQ integration tests to Java 8 for two reasons:

  • We want to be sure that client code compiles with Java 8
  • We started to get bored of writing the same old loops over and over again

The trigger was a loop where we needed to transform a SQLDialect[] into another SQLDialect[] calling .family() on each array element. Consider:

Java 7

SQLDialect[] families = 
    new SQLDialect[dialects.length];
for (int i = 0; i < families.length; i++)
    families[i] = dialects[i].family();

Java 8

SQLDialect[] families = 
      .map(d ->

OK, it turns out that the two solutions are equally verbose, even if the latter feels a bit more elegant. 🙂

And this gets us straight into the next topic:


For backwards-compatibility reasons, arrays and the pre-existing Collections API have not been retrofitted to accommodate all the useful methods that Streams now have. In other words, an array doesn’t have a map() method, just as much as List doesn’t have such a method. Streams and Collections/arrays are orthogonal worlds. We can transform them into each other, but they don’t have a unified API.

This is fine in everyday work. We’ll get used to the Streams API and we’ll love it, no doubt. But because of Java being extremely serious about backwards compatibility, we will have to think about one or two things more deeply.

Recently, we have published a post about The Dark Side of Java 8. It was a bit of a rant, although a mild one in our opinion (and it was about time to place some criticism, after all the praise we’ve been giving Java 8 in our series, before 😉 ). First off, that post triggered a reaction by Edwin Dalorzo from our friends at Informatech. (Edwin has written this awesome post comparing LINQ and Java 8 Streams, before). The criticism in our article evolved around three main aspects:

  • Overloading getting more complicated (see also this compiler bug)
  • Limited support for method modifiers on default methods
  • Primitive type “API overloads” for streams and functional interfaces

A response by Brian Goetz

I then got a personal mail from no one less than Brian Goetz himself (!), who pointed out a couple of things to me that I had not yet thought about in this way:

I still think you’re focusing on the wrong thing. Its not really the syntax you don’t like; its the model — you don’t want “default methods”, you want traits, and the syntax is merely a reminder that you didn’t get the feature you wanted. (But you’d be even more confused about “why can’t they be final” if we dropped the “default” keyword!) But that’s blaming the messenger (where here, the keyword is the messenger.)

Its fair to say “this isn’t the model I wanted”. There were many possible paths in the forest, and it may well be the road not taken was equally good or better.

This is also what Edwin had concluded. Default methods were a necessary means to tackle all the new API needed to make Java 8 useful. If Iterator, Iterable, List, Collection, and all the other pre-existing interfaces had to be adapted to accommodate lambdas and Streams API interaction, the expert group would have needed to break an incredible amount of API. Conversely, without adding these additional utility methods (see the awesome new Map methods, for instance!), Java 8 would have been only half as good.

And that’s it.

Even if maybe, some more class building tools might have been useful, they were not in the center of focus for the expert group who already had a lot to do to get things right. The center of focus was to provide a means for API evolution. Or in Brian Goetz’s own words:

Reaching out to the community

It’s great that Brian Goetz reaches out to the community to help us get the right picture about Java 8. Instead of explaining rationales about expert group decisions in private messages, he then asked me to publicly re-ask my questions again on Stack Overflow (or lambda-dev), such that he can then publicly answer them. For increased publicity and greater community benefit, I chose Stack Overflow. Here are:

The amount of traction these two questions got in no time shows how important these things are to the community, so don’t miss reading through them!

“Uncool”? Maybe. But very stable!

Java may not have the “cool” aura that node.js has. You may think about JavaScript-the-language whatever you want (as long as it contains swear words), but from a platform marketing perspective, Java is being challenged for the first time in a long time – and being “uncool” and backwards-compatible doesn’t help keeping developers interested.

But let’s think long-term, instead of going with trends. Having such a great professional platform like the Java language, the JVM, the JDK, JEE, and much more, is invaluable. Because at the end of the day, the “uncool” backwards-compatibility can also be awesome. As mentioned initially, we have upgraded our integration tests to Java 8. Not a single compilation error, not a single bug. Using Eclipse’s BETA support for Java 8, I could easily transform anonymous classes into lambdas and write awesome things like these upcoming jOOQ 3.4 nested transactions (API not final yet):

ctx.transaction(c1 -> {
       .values(3, "Doe")

    // Implicit savepoint here
    try {
        DSL.using(c1).transaction(c2 -> {
               .set(AUTHOR.FIRST_NAME, "John")

            // Rollback to savepoint
            throw new MyRuntimeException("No");

    catch (MyRuntimeException ignore) {}

    return 42;

So at the end of the day, Java is great. Java 8 is a tremendous improvement over previous versions, and with great people in the expert groups (and reaching out to the community on social media), I trust that Java 9 will be even better. In particular, I’m looking forward to learning about how these two projects evolve:

Although, again, I am really curious how they will pull these two improvements off from a backwards-compatibility perspective, and what caveats we’ll have to understand, afterwards. 😉

Anyway, let’s hope the expert groups will continue to provide public feedback on Stack Overflow. Stay tuned for more awesome Java 8 content on this blog.

Java Rocks More Than Ever

On the TIOBE index, Java and C have been sharing the #1 and #2 rank for a long time now, and with the recent GA release of the JDK 8, things are not going to get any worse for our community.

Java simply rocks! And it’s the best platform to build almost any of your applications, out there.

But why does Java rock so much? Is it the JVM? Is it the backwards-compatibility? Is it the easy syntax? Or the millions of free and commercial software available to build your software? All of this and much more.

The Top 10 Reasons why Java Rocks More Than Ever

ZeroTurnaround’s RebelLabs often publish awesome blog posts, which we can only recommend. In this case, we’ve discovered a very well-written series of blog posts explaining why Java is so great in 10 steps, by ZeroTurnaround’s Geert Bevin. The articles include:

Part 1: The Java Compiler

The compiler is one of the things we take for granted in any language, without thinking about its great features. In Java, unlike C++, you can simply compile your code without thinking too much about linking, optimisation and all sorts of other usual compiler features. This is partially due to the JIT (Just In Time compiler), which does further compilation work at runtime.

Read the full article here

Part 2: The Core API

The JDK’s core API consists of a very solid, stable and well-understood set of libraries. While many people complain about the lack of functionality in this area (resorting to Google Guava or Apache Commons), people often forget that the core API is still the one that is underneath all those extensions. Again, from a C++ perspective, this is a truly luxurious situation.

Read the full article here

Part 3: Open Source

In this section, ZeroTurnaround’s Geert Bevin‘s mind-set aligns well with our own at Data Geekery when it comes to the spirit of Open Source – no matter whether this is about free-as-in-freedom, or free-as-in-beer, the point is that so many things about Java are “open”. We’re all in this together.

Read the full article here

Part 4: The Java Memory Model

Again, a very interesting point of view from someone with a solid C++ background. We’re taking many things for granted as Java has had a very good threading and memory model from the beginning, which was corrected only once in the JDK 1.5 in 2004, and which has built a solid grounds for newer API like actor-based ones, Fork/JOIN, etc.

Read the full article here

Part 5: High-Performance JVM

The JVM is the most obvious thing to talk about it has allowed for so many languages to work on so many hardware environments, and it runs so fast, nowadays!

Read the full article here

Part 6: Bytecode

… and the JVM also rocks because of bytecode, of course. Bytecode is a vendor-independent abstraction of machine code, which is very predictable and can be generated, manipulated, and transformed by various technologies. We’ve recently had a guest post by Dr. Ming-Yee Iu who has shown how bytecode transformations can be used to emulate LINQ in Java. Let’s hear it for bytecode!

Read the full article here

Part 7: Intelligent IDEs

15 years ago, developing software worked quite differently. People can write assembler or C programs with vi or Notepad. But when you’re writing a very complex enterprise-scale Java program, you wouldn’t want to miss IDEs, nowadays. We’ve blogged about various reasons why SQLJ has died. The lack of proper IDE support was one of them.

Read the full article here

Part 8: Profiling Tools

Remember when Oracle released Java Mission Control for free developer use with the JDK 7u40? Profiling is something very very awesome. With modern profilers, you can know exactly where your bottleneck is by simply measuring every aspect of your JVM. You don’t have to guess, you can know. How powerful is that?

Read the full article here

Part 9: Backwards Compatibility

While backwards-compatibility has its drawbacks, too, it is still very impressive how long the Java language, the JVM, and the JDK have existed so far without introducing any major backwards-compatibility regressions. The only thing that comes to mind is the introduction of keywords like assert and enum.

Could you imagine introducing the Java 8 Streams API, lambda expressions, default methods, generics, enums, and loads of other features without ever breaking anything? That’s just great!

Read the full article here

Part 10: Maturity With Innovation

In fact, this article is a summary of all the others, saying that Java has been a very well-designed and mature platform from the beginning without ever ceasing to innovate. And it’s true. With Java 8, a great next step has been published that will – again – change the way the enterprise perceives software development for good.

Read the full article here

Java Rocks More Than Ever

It does, and it’s a great great platform with a bright future for all its community participants.