You Will Regret Applying Overloading with Lambdas!

Writing good APIs is hard. Extremely hard. You have to think of an incredible amount of things if you want your users to love your API. You have to find the right balance between:

  1. Usefulness
  2. Usability
  3. Backward compatibility
  4. Forward compatibility

We’ve blogged about this topic before, in our article: How to Design a Good, Regular API. Today, we’re going to look into how…

Java 8 changes the rules


Overloading is a nice tool to provide covenience in two dimensions:

  • By providing argument type alternatives
  • By providing argument default values

Examples for the above from the JDK include:

public class Arrays {

    // Argument type alternatives
    public static void sort(int[] a) { ... }
    public static void sort(long[] a) { ... }

    // Argument default values
    public static IntStream stream(int[] array) { ... }
    public static IntStream stream(int[] array, 
        int startInclusive, 
        int endExclusive) { ... }

The jOOQ API is obviously full of such convenience. As jOOQ is a DSL for SQL, we might even abuse a little bit:

public interface DSLContext {
    <T1> SelectSelectStep<Record1<T1>> 
        select(SelectField<T1> field1);

    <T1, T2> SelectSelectStep<Record2<T1, T2>> 
        select(SelectField<T1> field1, 
               SelectField<T2> field2);

    <T1, T2, T3> SelectSelectStep<Record3<T1, T2, T3>> s
        select(SelectField<T1> field1, 
               SelectField<T2> field2, 
               SelectField<T3> field3);

    <T1, T2, T3, T4> SelectSelectStep<Record4<T1, T2, T3, T4>> 
        select(SelectField<T1> field1, 
               SelectField<T2> field2, 
               SelectField<T3> field3, 
               SelectField<T4> field4);

    // and so on...

Languages like Ceylon take this idea of convenience one step further by claiming that the above is the only reasonable reason why overloading is be used in Java. And thus, the creators of Ceylon have completely removed overloading from their language, replacing the above by union types and actual default values for arguments. E.g.

// Union types
void sort(int[]|long[] a) { ... }

// Default argument values
IntStream stream(int[] array,
    int startInclusive = 0,
    int endInclusive = array.length) { ... }

Read “Top 10 Ceylon Language Features I Wish We Had In Java” for more information about Ceylon.

In Java, unfortunately, we cannot use union types or argument default values. So we have to use overloading to provide our API consumers with convenience methods.

If your method argument is a functional interface, however, things changed drastically between Java 7 and Java 8, with respect to method overloading. An example is given here from JavaFX.

JavaFX’s “unfriendly” ObservableList

JavaFX enhances the JDK collection types by making them “observable”. Not to be confused with Observable, a dinosaur type from the JDK 1.0 and from pre-Swing days.

JavaFX’s own Observable essentially looks like this:

public interface Observable {
  void addListener(InvalidationListener listener);
  void removeListener(InvalidationListener listener);

And luckily, this InvalidationListener is a functional interface:

public interface InvalidationListener {
  void invalidated(Observable observable);

This is great, because we can do things like:

Observable awesome = 
awesome.addListener(fantastic -> splendid.cheer());

(notice how I’ve replaced foo/bar/baz with more cheerful terms. We should all do that. Foo and bar are so 1970)

Unfortunately, things get more hairy when we do what we would probably do, instead. I.e. instead of declaring an Observable, we’d like that to be a much more useful ObservableList:

ObservableList<String> awesome = 
awesome.addListener(fantastic -> splendid.cheer());

But now, we get a compilation error on the second line:

awesome.addListener(fantastic -> splendid.cheer());
//      ^^^^^^^^^^^ 
// The method addListener(ListChangeListener<? super String>) 
// is ambiguous for the type ObservableList<String>

Because, essentially…

public interface ObservableList<E> 
extends List<E>, Observable {
    void addListener(ListChangeListener<? super E> listener);


public interface ListChangeListener<E> {
    void onChanged(Change<? extends E> c);

Now again, before Java 8, the two listener types were completely unambiguously distinguishable, and they still are. You can easily call them by passing a named type. Our original code would still work if we wrote:

ObservableList<String> awesome = 
InvalidationListener hearYe = 
    fantastic -> splendid.cheer();


ObservableList<String> awesome = 
    fantastic -> splendid.cheer());

Or even…

ObservableList<String> awesome = 
awesome.addListener((Observable fantastic) -> 

All of these measures will remove ambiguity. But frankly, lambdas are only half as cool if you have to explicitly type the lambda, or the argument types. We have modern IDEs that can perform autocompletion and help infer types just as much as the compiler itself.

Imagine if we really wanted to call the other addListener() method, the one that takes a ListChangeListener. We’d have to write any of

ObservableList<String> awesome = 

// Agh. Remember that we have to repeat "String" here
ListChangeListener<String> hearYe = 
    fantastic -> splendid.cheer();


ObservableList<String> awesome = 

// Agh. Remember that we have to repeat "String" here
    fantastic -> splendid.cheer());

Or even…

ObservableList<String> awesome = 

// WTF... "extends" String?? But that's what this thing needs...
awesome.addListener((Change<? extends String> fantastic) -> 

Overload you shan’t. Be wary you must.

API design is hard. It was hard before, it has gotten harder now. With Java 8, if any of your API methods’ arguments are a functional interface, think twice about overloading that API method. And once you’ve concluded to proceed with overloading, think again, a third time whether this is really a good idea.

Not convinced? Have a close look at the JDK. For instance the type. How many overloaded methods do you see that have the same number of functional interface arguments, which again take the same number of method arguments (as in our previous addListener() example)?


There are overloads where overload argument numbers differ. For instance:

<R> R collect(Supplier<R> supplier,
              BiConsumer<R, ? super T> accumulator,
              BiConsumer<R, R> combiner);

<R, A> R collect(Collector<? super T, A, R> collector);

You will never have any ambiguity when calling collect().

But when the argument numbers do not differ, and neither do the arguments’ own method argument numbers, the method names are different. For instance:

<R> Stream<R> map(Function<? super T, ? extends R> mapper);
IntStream mapToInt(ToIntFunction<? super T> mapper);
LongStream mapToLong(ToLongFunction<? super T> mapper);
DoubleStream mapToDouble(ToDoubleFunction<? super T> mapper);

Now, this is super annoying at the call site, because you have to think in advance what method you have to use based on a variety of involved types.

But it’s really the only solution to this dilemma. So, remember:

You Will Regret Applying Overloading with Lambdas!

Did you like this article? You might also like:

How to Translate SQL GROUP BY and Aggregations to Java 8

I couldn’t resist. I have read this question by Hugo Prudente on Stack Overflow. And I knew there had to be a better way than what the JDK has to offer.

The question reads:

I’m looking for a lambda to refine the data already retrieved. I have a raw resultset, if the user do not change the date I want use java’s lambda to group by the results for then. And I’m new to lambdas with java.

The lambda I’m looking for works simliar to this query.

    z, w, 
    MIN(x), MAX(x), AVG(x), 
    MIN(y), MAX(y), AVG(y) 
FROM table 
GROUP BY z, w;

SQL is declarative. Functional programming is not.

Before we go on with this discussion, let’s establish a very important fact. SQL is a completely declarative language. Functional (or “functional-ish”, to keep the Haskell-aficionados at peace) programming languages like Java 8 are not declarative. While expressing data transformation algorithms using functions is much more concise than expressing them using objects, or worse, using imperative instructions, you’re still explicitly expressing the algorithm.

When you write SQL, you don’t write any algorithm. You just describe the result you want to have. The SQL engine’s optimiser will figure out the algorithm for you – e.g. based on the fact that you may have an index on Z but not on W or on (Z, W).

While simple examples like these can easily be implemented using Java 8, you will quickly run into Java’s limitations, once you need to do more complex reporting.

Of course, as we’ve blogged before, the optimum is reached when you combine SQL and functional programming.

How can this be written in Java 8?

There are a variety of ways to do it. The essence is to understand all the participants in such a transformation. And no matter if you find this easy or hard, suitable for Java 8 or inadequate, thinking about the different, lesser-known parts of new Stream API is certainly worth the exercise.

The main participants here are:

  • Stream: If you’re using JDK 8 libraries, then the new type will be your first choice.
  • Collector: The JDK provides us with a rather low-level and thus very powerful new API for data aggregation (also known as “reduction”). This API is summarised by the new type, a new type from which we have heard only little so far in the blogosphere


Some of the code displayed here might not work in your favourite IDE. Unfortunately, even if Java 7 reaches its end of life, all major IDEs (Eclipse, IntelliJ, NetBeans), and even the javac compiler still have quite a few bugs related to the combination of generic type inference and lambda expressions. Stay tuned until those bugs are fixed! And report any bug you discover. We’ll all thank you for it!

Let’s go!

Let’s review our SQL statement:

    z, w, 
    MIN(x), MAX(x), AVG(x), 
    MIN(y), MAX(y), AVG(y) 
FROM table 
GROUP BY z, w;

In terms of the Stream API, the table itself is the Stream. Let’s just assume that we have a “table type” A as such:

class A {
    final int w;
    final int x;
    final int y;
    final int z;

    A(int w, int x, int y, int z) {
        this.w = w;
        this.x = x;
        this.y = y;
        this.z = z;

    public String toString() {
        return "A{" +
                "w=" + w +
                ", x=" + x +
                ", y=" + y +
                ", z=" + z +

You can also add equals() and hashCode() if you must.

We can now easily compose the Stream using Stream.of(), and some sample data:

Stream<A> stream =
    new A(1, 1, 1, 1),
    new A(1, 2, 3, 1),
    new A(9, 8, 6, 4),
    new A(9, 9, 7, 4),
    new A(2, 3, 4, 5),
    new A(2, 4, 4, 5),
    new A(2, 5, 5, 5));

Now, the next step is to GROUP BY z, w. The Stream API itself, unfortunately, doesn’t contain such a convenience method. We have to resort to more low-level operations by specifying the more general Stream.collect() operation, and passing a Collector to it that does the grouping. Luckily, a variety of different grouping Collectors are already made available from the Collectors helper class.

So we add that to our stream

    new A(1, 1, 1, 1),
    new A(1, 2, 3, 1),
    new A(9, 8, 6, 4),
    new A(9, 9, 7, 4),
    new A(2, 3, 4, 5),
    new A(2, 4, 4, 5),
    new A(2, 5, 5, 5))

jool-logo-blackNow the interesting part starts. How do we specify that we want to group by both A.z and A.w? We need to provide this groupingBy method with a function that can extract something like a SQL tuple from the A type. We could write our own tuple, or we simply use that of jOOλ, a library that we have created and open-sourced to improve our jOOQ integration tests.

The Tuple2 type roughly looks like this:

public class Tuple2<T1, T2> {

    public final T1 v1;
    public final T2 v2;

    public T1 v1() {
        return v1;

    public T2 v2() {
        return v2;

    public Tuple2(T1 v1, T2 v2) {
        this.v1 = v1;
        this.v2 = v2;

public interface Tuple {
    static <T1, T2> Tuple2<T1, T2> tuple(T1 v1, T2 v2) {
        return new Tuple2<>(v1, v2);

It has many more useful features, but these ones will be sufficient for this article.

On a side-note

Why the JDK doesn’t ship with built-in tuples like C#’s or Scala’s escapes me.

Functional programming without tuples is like coffee without sugar: A bitter punch in your face.

Anyway… back on track

So we’re grouping by the (A.z, A.w) tuple, as we would in SQL

Map<Tuple2<Integer, Integer>, List<A>> map =
    new A(1, 1, 1, 1),
    new A(1, 2, 3, 1),
    new A(9, 8, 6, 4),
    new A(9, 9, 7, 4),
    new A(2, 3, 4, 5),
    new A(2, 4, 4, 5),
    new A(2, 5, 5, 5))
    a -> tuple(a.z, a.w)

As you can see, this produces a verbose but very descriptive type, a map containing our grouping tuple as its key, and a list of collected table records as its value.

Running the following statement


will yield:

(1, 1)=[A{w=1, x=1, y=1, z=1}, A{w=1, x=2, y=3, z=1}]
(4, 9)=[A{w=9, x=8, y=6, z=4}, A{w=9, x=9, y=7, z=4}]
(5, 2)=[A{w=2, x=3, y=4, z=5}, A{w=2, x=4, y=4, z=5}, A{w=2, x=5, y=5, z=5}]

That’s already quite awesome! In fact, this behaves like the SQL:2011 standard COLLECT() aggregate function, that is also available in Oracle 10g+

Now, instead of actually collecting the A records, we prefer to aggregate the individual values of x and y. The JDK provides us with a couple of interesting new types, e.g. the java.util.IntSummaryStatistics, which is available for convenience again from the Collectors type via Collectors.summarizingInt().

On a side note

For my taste, this sledge-hammer data aggregation technique is a bit quirky. The JDK libraries have been left intentionally low level and verbose, perhaps to keep the library footprint small, or to prevent “horrible” consequences when in 5-10 years (after the release of JDK 9 and 10), it becomes obvious that some features may have been added prematurely.

At the same time, there is this all-or-nothing IntSummaryStatistics, that blindly aggregates these popular aggregation values for your collection:

  • COUNT(*)
  • SUM()
  • MIN()
  • MAX()

and obviously, once you have SUM() and COUNT(*), you also have AVG() = SUM() / COUNT(*). So that’s going to be the Java way. IntSummaryStatistics.

In case you were wondering, the SQL:2011 standard specifies these aggregate functions:


And obviously there are many other, vendor-specific aggregate and window functions in SQL. We’ve blogged about them all:

True, MIN, MAX, SUM, COUNT, AVG are certainly the most popular ones. But it would’ve been nicer if they hadn’t been included in these default aggregation types, but made available in a much more composable way.

Anyway… back on track

If you want to stay low-level and use mostly JDK API, you can use the following technique to implement aggregation over two columns:

    Tuple2<Integer, Integer>, 
    Tuple2<IntSummaryStatistics, IntSummaryStatistics>
> map = Stream.of(
    new A(1, 1, 1, 1),
    new A(1, 2, 3, 1),
    new A(9, 8, 6, 4),
    new A(9, 9, 7, 4),
    new A(2, 3, 4, 5),
    new A(2, 4, 4, 5),
    new A(2, 5, 5, 5))
    a -> tuple(a.z, a.w),

        // When collecting, we'll aggregate data
        // into two IntSummaryStatistics for x and y
        () -> tuple(new IntSummaryStatistics(), 
                    new IntSummaryStatistics()),

        // The accumulator will simply take
        // new t = (x, y) values
        (r, t) -> {

        // The combiner will merge two partial
        // aggregations, in case this is executed
        // in parallel
        (r1, r2) -> {

            return r1;


The above would now print

(1, 1)=(IntSummaryStatistics{count=2, sum=3, min=1, average=1.500000, max=2}, 
        IntSummaryStatistics{count=2, sum=4, min=1, average=2.000000, max=3})
(4, 9)=(IntSummaryStatistics{count=2, sum=17, min=8, average=8.500000, max=9}, 
        IntSummaryStatistics{count=2, sum=13, min=6, average=6.500000, max=7})
(5, 2)=(IntSummaryStatistics{count=3, sum=12, min=3, average=4.000000, max=5}, 
        IntSummaryStatistics{count=3, sum=13, min=4, average=4.333333, max=5})

But obviously, no one will want to write that much code. The same thing can be achieved with jOOλ with much less code

    Tuple2<Integer, Integer>, 
    Tuple2<IntSummaryStatistics, IntSummaryStatistics>
> map =

// Seq is like a Stream, but sequential only,
// and with more features
    new A(1, 1, 1, 1),
    new A(1, 2, 3, 1),
    new A(9, 8, 6, 4),
    new A(9, 9, 7, 4),
    new A(2, 3, 4, 5),
    new A(2, 4, 4, 5),
    new A(2, 5, 5, 5))

// Seq.groupBy() is just short for 
// Stream.collect(Collectors.groupingBy(...))
    a -> tuple(a.z, a.w),

    // ... because once you have tuples, 
    // why not add tuple-collectors?
        Collectors.summarizingInt(a -> a.x),
        Collectors.summarizingInt(a -> a.y)

What you see above is probably as close as it gets to the original, very simmple SQL statement:

    z, w, 
    MIN(x), MAX(x), AVG(x), 
    MIN(y), MAX(y), AVG(y) 
FROM table 
GROUP BY z, w;

The interesting part here is the fact that we have what we call “tuple-collectors”, a Collector that collects data into tuples of aggregated results for any degree of the tuple (up to 8). Here’s the code for Tuple.collectors:

// All of these generics... sheesh!
static <T, A1, A2, D1, D2> 
       Collector<T, Tuple2<A1, A2>, Tuple2<D1, D2>> 
    Collector<T, A1, D1> collector1
  , Collector<T, A2, D2> collector2
) {
    return Collector.of(
        () -> tuple(
          , collector2.supplier().get()
        (a, t) -> {
            collector1.accumulator().accept(a.v1, t);
            collector2.accumulator().accept(a.v2, t);
        (a1, a2) -> tuple(
            collector1.combiner().apply(a1.v1, a2.v1)
          , collector2.combiner().apply(a1.v2, a2.v2)
        a -> tuple(
          , collector2.finisher().apply(a.v2)

Where the Tuple2<D1, D2> is the aggregation result type that we derive from collector1 (which provides D1) and from collector2 (which provides D2).

That’s it. We’re done!


Java 8 is a first step towards functional programming in Java. Using Streams and lambda expressions, we can already achieve quite a bit. The JDK APIs, however, are extremely low level and the experience when using IDEs like Eclipse, IntelliJ, or NetBeans can still be a bit frustrating. While writing this article (and adding the Tuple.collectors() method), I have reported around 10 bugs to the different IDEs. Some javac compiler bugs are not yet fixed, prior to JDK 1.8.0_40 ea. In other words:

I just keep throwing generic type parameters at the darn thing until the compiler stops bitching at me

But we’re on a good path. I trust that more useful API will ship with JDK 9 and especially with JDK 10, when all of the above will hopefully profit from the new value types and generic type specialization.

jool-logo-blackAnd, of course, if you haven’t already, download and contribute to jOOλ here!

We have created jOOλ to add the missing pieces to the JDK libraries. If you want to go all in on functional programming, i.e. when your vocabulary includes hipster terms (couldn’t resist) like monads, monoids, functors, and all that, we suggest you skip the JDK’s Streams and jOOλ entirely, and go download functionaljava by Mark Perry or javaslang by Daniel Dietrich

jOOQ Newsletter: January 21, 2015 – Groovy and Open Source – jOOQ and the strong Swiss Franc

Subscribe to this newsletter here

Tweet of the Day

Today, we’re very happy to have “spied” on our users as we can now show you a whole Tweet Conversation of the Day

RxJooq, or reactive jOOQ. How does that sound!? Yes, jOOQ is growing to become a hype among SQL and fluent API aficionados. A recent discussion on reddit already puts jOOQ on the same level with Hibernate with more than 10 mentions in answers to the question “Java: What ORM to use”. Our goal has always been for a Java developer to ask themselves at the beginning of a project:

Is this a jOOQ project, or is this a Hibernate project (or both)?

It is too early to announce anything, but at Data Geekery, we’re very interested and thus putting efforts into collaborating with Red Hat to make the jOOQ / Hibernate integration work more seamlessly, so stay tuned for more goodness in that area.

Groovy and Open Source – What it means for us

You may have heard of Pivotal’s recent announcement about their withdrawing sponsorship from the Groovy and Grails ecosystem. This isn’t exactly a surprise to many people as Pivotal’s main focus has shifted towards their PaaS business quite some time ago. The interesting aspect from our perspective is the fact that a whole ecosystem seems to have relied on the benevolence of a single sponsor. Quite a risk!

We think that Open Source should work differently. Open Source is a fine means of offering freemium and (legally) riskless software to potential customers in order to help customers start engaging with a brand. The ultimate vendor goal with Open Source is always upselling. As our valued jOOQ users and jOOQ newsletter and blog readers, we obviously hope that you will eventually understand all the combined SQL value put into jOOQ, and thus upgrade to a commercial jOOQ subscription.

This wasn’t necessarily the case at Pivotal. There is no obvious path from using Groovy (or Grails) to buying Pivotal’s cloud platform solutions. To make things worse, in order to survive, the Groovy platform now depends on a new, arbitrary sponsor whose incentive to sponsor Groovy might be 100% different from Pivotal’s. For the end user, this will not be the same Groovy any more – so it is hard to believe that Groovy will not suffer heavily from any future transition.

We believe that vendors shouldn’t depend on benevolence. We believe that vendors should have a very clear strategy why they’re creating a product, and do everything necessary to satisfy real customer’s needs. So we want to take the opportunity and thank you for being with us, and for making jOOQ (both the Open Source Edition and the Commercial Editions) what it is: A platform valued by both users of Open Source and commercial databases.

More information about our take on Pivotal and Groovy can be found on our blog:

It’s jOONuary! Profit from our 20% Discount Promotion

Speaking of our customers, there has never been a better time to become one!

Your budget for 2015 has been set in stone? You spent too much money on geeky infrastructure during the Holiday Season? Not a problem for your planned jOOQ integration! If you purchase new jOOQ licenses in jOONuary (January 2015), we will offer you a limited-time 20% discount on all price plans. Act quickly!

jOOQ and the Strong Swiss Franc

We’re a Switzerland-based company, and as such are heavily influenced by recent events on the currency exchange markets. The EUR (which is our sale currency) has plummeted almost 20% compared to the CHF (which is our accounting currency).

This affects all of the Swiss export industry, and many companies are starting to take measures. We will not take any measures thus far and continue with our existing EUR-based price model. For our international customers, nothing will change. For our Swiss customers, this means that in addition to the above jOONuary discount, you will now also benefit from a “Euro discount”! Did we say there has never been a better time to become our customer?

jOOQ 3.6 Outlook

The upcoming jOOQ 3.6 will not be less exciting than the previous versions in the least bit. Here is a quick outline of what we’re going to be doing in the upcoming release:

  • SAP HANA support. We’ve been talking to database vendors in the past, and we continue to do so, maintaining good relationships with the technical and community people at the vendor side. This time, the collaboration initiative came from the vendor directly, and we’ve heard them.

    SAP HANA is an emerging cloud SQL and in-memory SQL platform, with a big Java and Scala based tool chain, which constitutes a perfect match for the jOOQ ecosystem. We’re going to support both HANA’s SQL features as well as HANA’s SQLScript features in the jOOQ 3.6 Enterprise Edition. If you’re an SAP HANA user and interested in details, or in a free preview of jOOQ 3.6.0, please contact sales right away. We’re more than happy to provide you with more info.

  • Nested records and tables. One of the SQL standard’s most underestimated features is the capability of nesting records and tables. In a true ORDBMS, tables (or MULTISETs) can be nested any level deep. If your SQL database supports these features, it is very easy to materialise a nested object graph directly in the database, instead of relying on the JOIN-based workarounds provided by modern ORMs.

    Nesting of records can also be very useful when reusing common data structures, such as audit columns (creation_date, creation_user, modification_date, modification_user). JPA supports the @Embedded annotation for this, and we’ll delve into these features as well.

    We believe that true MULTISET support will obsolete our competing products’ most important asset: mapping. Once you can declare all mapping already in SQL, you will no longer miss JPA once you’ve migrated to jOOQ.

  • A new ConverterProvider SPI. Converters are great for supporting custom data types, but having to register them all the time is tedious. What if jOOQ just supported T <-> U conversion right out of the box, for any combination of T and U? We’ll let you register all your favourite converters and jOOQ figures out the conversion path through the converter graph.
  • Even better PL/SQL support. PL/SQL types are ubiquitous, but they are not easily accessible via JDBC, and thus via jOOQ. We’re researching a variety of possibilities of working around JDBC’s limitations to allow you to use your favourite PL/SQL types: BOOLEAN, RECORD types, perhaps even table types.


Upcoming jOOQ Events

Have you missed one of our talks and presentations in the recent past? No problem at all, we’re back on the road after a short winter break. Here are all of our upcoming events:

Keep up to date with our own and third-party jOOQ events on our news website:

We’re looking forward to meeting you and to talking about all things Java and SQL!

Open Source Doesn’t Need More Support. It Needs Better Business Models

Jamie Allen, Typesafe‘s Director of Global Services published an interesting point of view on Twitter:

And he’s right of course. We are constantly reminded of the fact that we should support FOSS projects on which we depend. Just recently, Wikipedia had this huge banner on top of it, asking for money, and we probably should. But do we really depend on them, and can we really make a difference? Let us look at the problem from a business perspective.

There isn’t a second Red Hat

About a year ago, there was an extremely interesting article on Tech Crunch by Peter Levine, partner at Andreessen Horowitz who have just invested $40M in Stack Exchange. The article was about Why There Will Never Be Another RedHat: The Economics Of Open Source. It compared Red Hat’s and VMWare’s market capitalisation and revenue with that of Microsoft, Oracle, or Amazon, showing that even Red Hat is a rather insignificant competitor in terms of these size metrics.

Why is this so?

Let’s go back to Groovy: There are probably tens of thousands of Groovy developers out there who have simply downloaded Groovy and then never again interacted with the vendor. It probably wouldn’t even be wrong to say that many developers weren’t aware of Pivotal having been the main sponsor behind Groovy. Sure, Groovy is a strong brand, but it is really “everybody’s brand”, and thus: nobody’s brand. Not being a strong brand, it attracted only techies with language interests (and it is a beautiful language to work with, indeed!)

Now, Pivotal has withdrawn their engagement from Groovy, for completely understandable reasons. Let’s review Jamie’s point of view:

Pivotal’s move to end support of Groovy is a stark reminder that enterprises who depend on FOSS projects should help support them.

Would it have mattered if “we” had supported Groovy?

Perhaps. A Groovy Foundation (similar to the Apache Foundation, or the Eclipse Foundation) might have made Groovy a bit less dependent on Pivotal, and this can still happen. Possibly, a couple of larger companies who depend on Groovy, or Gradle might chime in and become Silver or Gold or Platinum Sponsors, or something like that. Perhaps, Gradleware will seize the opportunity and “buy” Groovy to become THE Groovy company.

But will it work? Does the same work for Typesafe? Can monetising an Open Source language and platform work in times when even C# is now given away for free?

Red Hat can make money off Linux, because Linux is a very complex ecosystem that just requires a support subscription when you’re running it in production. No bank on this planet will ever run a server farm without the vendor promising 24h support with under 1h reaction time. But is the same true for Scala, Groovy? The “critical” work has long been done by the developers. The binaries are built and shipped onto production where operations takes over. Operations couldn’t care less if that binary is built with Groovy, Scala, Java, Ceylon, Kotlin, Fantom, or any of the other gazillion Java alternatives. All operations will ever care about is the JVM, or Weblogic – and the database, of course. And operations is where the long-term subscription money is, not development.

This doesn’t mean that no one should make money from developers. Companies like JetBrains, ZeroTurnaround, or also ourselves, Data Geekery show that it works, on a much smaller scale. But if a company is “selling” a programming language that doesn’t immediately help them upsell their customers to buy their significant other subscriptions, you should be wary as the vendor motivation to produce the programming language product is very unclear – and in the case of Pivotal, “unclear” is not even close to describe the vendor motivation.

Good examples of holistic platform strategies are these, because operations and the end user can immediately drive the decision chain that justifies the language lock-in for the developers:

  • C# -> Visual Studio -> SQL Server -> Azure, etc.
  • Java -> JVM / Weblogic -> Oracle Database -> Oracle Commerce, etc.

OK examples are these, although the upselling potential might not be viable enough to maintain a whole ecosystem. We’ll see how it works:

  • Kotlin -> IntelliJ

Less good examples are these, because the value proposition chain is really not obvious. There is no justification for the language lock-in:

  • Groovy -> Cloud Platform ??
  • Scala -> Reactive Programming ??
  • Ceylon -> RHEL ??

The Business Model

Jamie Allen’s Tweet shows a lot about what’s wrong with many Open Source vendors. While he claims that end users depend on OSS products from their vendors, the opposite is true. The end user can simply fork the OSS product and lead it to a graceful end of life before replacing it. But the vendor really depends on the goodwill and the benevolence of their FOSS communities. The vendor then tries to leverage goodwill to make a weird-sounding upselling between completely unrelated products. This cannot work.

So join us in our endeavours. Make Open Source a business. A viable business, a business driven by the vendor (and by the market, of course). A business that makes sense. A business that involves dual-licensing and reasonable upselling. A business that uses Open Source mainly as a freemium entry point for the actual business.

You can be romantic about F(L)OSS in your heart, that’s OK. But please don’t depend on it. It would be too bad if you don’t succeed, just because you ran out of money from your “sponsors”, because you didn’t care about the business aspect of your product.

Read also 5 lessons for any open source business transitioning to a revenue-based model

Suis-je Groovy? No! What Pivotal’s Decision Means for Open Source Software

Today there was great news in the JVM ecosystem.

Pivotal, the company who is committed to OSS has become a bit less committed:


The reaction in the community were largely summarised by the hashtag #jesuisgroovy:

The interesting part in Pivotal’s announcement is this one:

The decision to conclude its sponsorship of Groovy and Grails is part of Pivotal’s larger strategy to concentrate resources on accelerating both commercial and open source projects that support its growing traction in Platform-as-a-Service, Data, and Agile development. Pivotal has determined that the time is right to let further development of Groovy and Grails be led by other interested parties in the open source community who can best serve the goals of those projects.

The official announcement can be read here.

Groovy is not a viable business (for Pivotal)

In other words, Groovy is not a viable business for Pivotal. And it’s hard to disagree here. Groovy has never been created with any commercial interests. Like many Open Source projects, Groovy was created in order to make something “better”, mostly for the sake of it being better. Of course it was useful as it introduced a lot of nice features into the Java ecosystem at a time before all these new JVM languages popped up. And before all these new JVM languages finally had an effect on Java-the-language itself.

On the other hand, the Groovy website’s rather geeky look-and-feel has never made it seem as though virtually anyone had any commercial interests in the language or the platform for that matter. I’m not trying to be harsh here, Groovy is an awesome language, created with love. But maintaining an Open Source ecosystem is hard work. It costs a lot of money and effort. And in the case of Groovy, it is just very hard to disagree with the fact that there is probably little money to be made out of it.

How to make money out of Open Source

When we moved on from a purely Open Source jOOQ to a dual-licensed one, we were criticised a lot by people who realised that they might fall into the dual-licensing category who no longer gets to ride for free. This was of course a disappointing evolution for those people. We see it as one step forward for a product that doesn’t just want to implement l’art pour l’art. We believe that we’re adding value on a small scale to a select set of customers with real SQL problems, and we want to continue to do so. Thus, commercial interests are now the driving force behind our developments, and dual-licensing is the easiest way to achieve that on our own small scale.

Many of those who had criticised us claimed that we should create a support-based Open Source business model instead (like Pivotal!). In other words: Let “them” pay for support – whoever “they” are. But that is not a viable model in the long run.

We create fishing poles. We don’t want to compete with our customers, the fishermen.

In software, the vendor of some product shouldn’t commoditize the main driver for innovation: The product. They should sell the product and create an ecosystem and a market for consultants that will be much better at applying the product to some concrete customer’s business. It is a win-win-win situation for everyone:

  • Vendors get money from licenses
  • Consultants get money from their specialist knowledge
  • End-users get a better, cheaper solution with a lower cost of ownership

Although, the consultant is always the one whose work is commoditized in the long run as demand for the product increases, and more consultants pop up trying to make money from their consulting business.

Joel Spolsky has written an extremely interesting Strategy Letter on the idea of commoditizing a complementary product (support, in this case) to increase the demand for the primary product (license, in this case). In the case of a PaaS company like Pivotal, however, we can only guess that even the commoditization of a whole programming language and ecosystem is no longer sustainable enough to increase demand for their PaaS offerings.

If that is the truth, then other platforms like Spring are at the stake as well! If Groovy is not sustainable, why should Spring be?

Am I Groovy?

Suis-je Groovy? To get back to the original claim:

No, I’m not Groovy. Groovy and every other piece of Open Source software that does not in any direct way produce a commercial value for both the vendor and the consumer is doomed to fail in the long run. Open Source software has created a tremendous amount of value in our industry. Companies like ourselves wouldn’t be possible if we’d still pay millions for an operating system license. The question is not whether software is free as in beer or as in freedom. The question is whether anyone has any viable commercial interests in making a particular software element cheap or even free. If they don’t, well, the joke will be on you as the vendor might just stop doing it. Open Source or not.

Using Java 8 to Prevent Excessively Wide Logs

Some logs are there to be consumed by machines and kept forever.

Other logs are there just to debug and to be consumed by humans. In the latter case, you often want to make sure that you don’t produce too much logs, especially not too wide logs, as many editors and other tools have problems once line lenghts exceed a certain size (e.g. this Eclipse bug).

String manipulation used to be a major pain in Java, with lots of tedious-to-write loops and branches, etc. No longer with Java 8!

The following truncate method will truncate all lines within a string to a certain length:

public String truncate(String string) {
    return truncate(string, 80);

public String truncate(String string, int length) {
    return Seq.of(string.split("\n"))
              .map(s -> StringUtils.abbreviate(s, 400))

The above example uses jOOλ 0.9.4 and Apache Commons Lang, but you can achieve the same using vanilla Java 8, of course:

public String truncate(String string) {
    return truncate(string, 80);

public String truncate(String string, int length) {
    return Stream.of(string.split("\n"))
                 .map(s -> s.substring(0, Math.min(s.length(), length)))

The above when truncating logs to length 10, the above program will produce:


Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu
fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
culpa qui officia deserunt mollit anim id est laborum.


Lorem ipsum dolor...
incididunt ut lab...
nostrud exercitat...
Duis aute irure d...
fugiat nulla pari...
culpa qui officia...

Happy logging!

Infinite Loops. Or: Anything that Can Possibly Go Wrong, Does.

A wise man once said:

Anything that can possibly go wrong, does


Some programmers are wise men, thus a wise programmer once said:

A good programmer is someone who looks both ways before crossing a one-way street.

Doug Linder

In a perfect world, things work as expected and you may think that it is a good idea to keep consuming things until the end. So the following pattern is found all over in every code base:


for (;;) {
    // something


while (1) {
    // something


10 something
20 GOTO 10

Want to see proof? Search github for while(true) and check out the number of matches:

Never use possibly infinite loops

There is a very interesting discussion in computer science around the topic of the “Halting Problem”. The essence of the halting problem as proved by Alan Turing a long time ago is the fact that it is really undecidable. While humans can quickly assess that the following program will never stop:

for (;;) continue;

… and that the following program will always stop:

for (;;) break;

… computers cannot decide on such things, and even very experienced humans might not immediately be able to do so when looking at a more complex algorithm.

Learning by doing

In jOOQ, we have recently learned about the halting problem the hard way: By doing.

Before fixing issue #3696, we worked around a bug (or flaw) in SQL Server’s JDBC driver. The bug resulted in SQLException chains not being reported correctly, e.g. when the following trigger raises several errors:


    Raiserror('Employee_Upd_2 Trigger called...',16,-1)
    Raiserror('Employee_Upd_2 Trigger called...1',16,-1)
    Raiserror('Employee_Upd_2 Trigger called...2',16,-1)
    Raiserror('Employee_Upd_2 Trigger called...3',16,-1)
    Raiserror('Employee_Upd_2 Trigger called...4',16,-1)
    Raiserror('Employee_Upd_2 Trigger called...5',16,-1)


So, we explicitly consumed those SQLExceptions, such that jOOQ users got the same behaviour for all databases:

consumeLoop: for (;;)
    try {
        if (!stmt.getMoreResults() && 
             stmt.getUpdateCount() == -1)
            break consumeLoop;
    catch (SQLException e) {
        previous = e;

This has worked for most of our customers, as the chain of exceptions thus reported is probably finite, and also probably rather small. Even the trigger example above is not a real-world one, so the number of actual errors reported might be between 1-5.

Did I just say … “probably” ?

As our initial wise men said: The number might be between 1-5. But it might just as well be 1000. Or 1000000. Or worse, infinite. As in the case of issue #3696, when a customer used jOOQ with SQL Azure. So, in a perfect world, there cannot be an infinite number of SQLException reported, but this isn’t a perfect world and SQL Azure also had a bug (probably still does), which reported the same error again and again, eventually leading to an OutOfMemoryError, as jOOQ created a huge SQLException chain, which is probably better than looping infinitely. At least the exception was easy to detect and work around. If the loop ran infinitely, the server might have been completely blocked for all users of our customer.

The fix is now essentially this one:

consumeLoop: for (int i = 0; i < 256; i++)
    try {
        if (!stmt.getMoreResults() && 
             stmt.getUpdateCount() == -1)
            break consumeLoop;
    catch (SQLException e) {
        previous = e;

True to the popular saying:

640 KB ought to be enough for anybody

The only exception

So as we’ve seen before, this embarassing example shows that anything that can possibly go wrong, does. In the context of possibly ininite loops, beware that this kind of bug will take entire servers down.

The Jet Propulsion Laboratory at the California Institute of Technology has made this an essential rule for their coding standards:

Rule 3 (loop bounds)

All loops shall have a statically determinable upper-bound on the maximum number of loop iterations. It shall be possible for a static compliance checking tool to affirm the existence of the bound. An exception is allowed for the use of a single non-terminating loop per task or thread where requests are received and processed. Such a server loop shall be annotated with the C comment: /* @non-terminating@ */.

So, apart from very few exceptions, you should never expose your code to the risk of infinite loops by not providing upper bounds to loop iterations (the same can be said about recursion, btw.)


Go over your code base today and look for any possible while (true), for (;;), do {} while (true); and other statements. Review those statements closely and see if they can halt – e.g. using break, or throw, or return, or continue (an outer loop).

Chances are, that you or someone before you who wrote that code was as naive as we were, believing that…

… oh come on, this will never happen

Because, you know what happens when you think that nothing will happen.