(Ab)using Java 8 FunctionalInterfaces as Local Methods


If you’re programming in more advanced languages like Scala or Ceylon, or even JavaScript, “nested functions” or “local functions” are a very common idiom to you. For instance, you’ll write things like fibonacci functions as such:

def f() = {
  def g() = "a string!"
  g() + "– says g"
}

(Question from Stack Overflow by Aaron Yodaiken)

The f() function contains a nested g() function, which is local to the scope of the outer f() function.

In Java, there is no way to create a local function like this, but you can assign a lambda expression to a local variable and use that instead.

The above example can be translated to the following Java code:

String f() {
    Supplier<String> g = () -> "a string!";
    return g.get() + "- says g";
}

While this example is rather trivial, a much more useful use-case is testing. For instance, consider the following jOOλ unit test, which checks whether the Stream.close() semantics is properly implemented across all sorts of jOOλ Seq methods, that combine two streams into one:

@Test
public void testCloseCombineTwoSeqs() {
    Consumer<BiFunction<Stream<Integer>, Stream<Integer>, Seq<?>>> test = f -> {
        AtomicBoolean closed1 = new AtomicBoolean();
        AtomicBoolean closed2 = new AtomicBoolean();
        
        Stream s1 = Stream.of(1, 2).onClose(() -> closed1.set(true));
        Stream s2 = Stream.of(3).onClose(() -> closed2.set(true));
        
        try (Seq s3 = f.apply(s1, s2)) {
            s3.collect(Collectors.toList());
        }

        assertTrue(closed1.get());
        assertTrue(closed2.get());
    };
    
    test.accept((s1, s2) -> seq(s1).concat(s2));
    test.accept((s1, s2) -> seq(s1).crossJoin(s2));
    test.accept((s1, s2) -> seq(s1).innerJoin(s2, (a, b) -> true));
    test.accept((s1, s2) -> seq(s1).leftOuterJoin(s2, (a, b) -> true));
    test.accept((s1, s2) -> seq(s1).rightOuterJoin(s2, (a, b) -> true));
}

The local function is test, and it takes two Stream<Integer> arguments, producing a Seq<?> result.

Why not just write a private method?

Of course, this could have been solved with a private method as well, classic Java style. But sometimes, using a local scope is much more convenient, as the test Consumer (local function) does not escape the scope of this single unit test. It should be used only within this single method.

An alternative, more classic Java way would have been to define a local class, instead, and put the function inside of that. But this solution is much more lean.

One disadvantage, however, is that recursion is much harder to implement this way, in Java. See also:
http://stackoverflow.com/q/19429667/521799

A Very Peculiar, but Possibly Cunning Kotlin Language Feature


This has caught me by surprise. After studying the Kotlin language to learn about how to best leverage this interesting new language for jOOQ, I stumbled upon this puzzler. What do you think the following program will print?

fun main(args: Array) {
    (1..5).forEach {
        if (it == 3)
            return
        print(it)
    }

    print("done")
}

Well… You might have guessed wrong. The above will print:

12

It will NOT print what most people might expect:

1245done

Note to those of you who are not surprised:

The above is peculiar for someone used to working with Java 8, where the following code will indeed print 1245done:

public static void main(String[] args) {
    IntStream.rangeClosed(1, 5).forEach(it -> {
        if (it == 3)
            return;

        System.out.print(it);
    });

    System.out.print("done");
}

The syntactical reason is explained in this section of the Kotlin manual:
https://kotlinlang.org/docs/reference/returns.html

In lambdas / closures, the return statement will not (necessarily) return from the lambda / closure, but from the immediate enclosing scope of the lambda / closure. The rationale has been kindly given to me by Dmitry Jemerov from JetBrains in two tweets:

Cunningly, the Kotlin language has removed language-based support for Java constructs like try-with-resources, or the synchronized statement. That’s very reasonable, because these language constructs don’t necessarily belong in the language (as we’ve previously claimed in another blog post), but could be moved to libraries instead. For example:

// try-with-resources is emulated using an
// extension function "use"
OutputStreamWriter(r.getOutputStream()).use {
    it.write('a')
}

(criticism here)

Or:

// Synchronized is a function!
val x = synchronized (lock, { computation() })

See also:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/synchronized.html

After all, even in Java, the language feature only works because the language depends on library types, like Iterable (foreach), AutoCloseable (try-with-resources), or JVM features (monitor on each reference for synchronized)

So, what’s the deal with return?

Along the lines of the above rationale, when language designers want to avoid language constructs for things that can be implemented with libraries, but still want you to feel like these were language constructs, then the only reasonable meaning of return inside of such a “construct-ish” lambda / closure is to return from the outer scope. So, when you write something like:

fun main(args : Array) {
    val lock = Object()
    val x = synchronized(lock, {
        if (1 == 1)
            return

        "1"
    })

    print(x)
}

The real intention is for this to be the equivalent of the following Java code:

public static void main(String[] args) {
    Object lock = new Object();
    String x;

    synchronized (lock) {
        if (1 == 1)
            return;

        x = "1";
    }

    System.out.println(x);
}

In the Java case, obviously, the return statement exits the main() method, because there is no other reasonable stack frame to return from. Unlike in Kotlin, where one might argue the lambda / closure would produce its own stack frame.

But it really doesn’t. The reason for this is the inline modifier on the synchronized function:

public inline fun <R> synchronized(lock: Any, block: () -> R): R {
    monitorEnter(lock)
    try {
        return block()
    }
    finally {
        monitorExit(lock)
    }
}

See also:
https://kotlinlang.org/docs/reference/inline-functions.html

Which means that the block closure passed as an argument isn’t really a pure lambda expression, but just syntactic sugar embedded in the call-site’s scope.

Weird. Cunning. Clever. But a bit unexpected.

Is this a good idea? Or will the language designers regret this, later on? Are all lambdas / closures potentially “language construct-ish”, where such a return statement is expected to leave the outer scope? Or are there clear cases where this inline behaviour just makes total sense?

We’ll see. In any case, it is very interesting for a language to have chosen this path.

Liked this article?

Stay tuned for a follow-up about the exciting Kotlin language. In the meantime, read

The Mute Design Pattern


Have you been writing a lot of code following the Mute-Design-Pattern™ lately? E.g.

try {
    complex();
    logic();
    here();
}
catch (Exception ignore) {
    // Will never happen hehe
    System.exit(-1);
}

There’s an easier way with Java 8!

Just add this very useful tool to your Utilities or Helper class:

public class Helper {

    // 18395 lines of other code here

    @FunctionalInterface
    interface CheckedRunnable {
        void run() throws Throwable;
    }

    public static void mute(CheckedRunnable r) {
        try {
            r.run();
        }
        catch (Throwable ignore) {
            // OK, better stay safe
            ignore.printStackTrace();
        }
    }

    // 37831 lines of other code here
}

Now you can wrap all your logic in this nice little wrapper:

mute(() -> {
    complex();
    logic();
    here();
});

Done!

Even better, in some cases, you can use method references

try (Connection con = ...;
     PreparedStatement stmt = ...) {

    mute(stmt::executeUpdate);
}

An Ingenious Workaround to Emulate Union Types in Java


Before I move on with the actual article, I’d like to give credit to Daniel Dietrich, author of the awesome Javaslang library, who has had the idea before me:

Contravariant Generic Bounds

It all started with a tweet:

I wanted to do something like pattern-matching a common super type of a set of types, along the lines of:

<T super T1 | T2 | ... | TN>

Note that what I really wanted is support for union types, not intersection types as I originally claimed.

Why did I want to do that? Because it would be a nice addition to the jOOλ library, which features typesafe tuples for Java:

class Tuple3<T1, T2, T3> {
    final T1 v1;
    final T2 v2;
    final T3 v3;

    // Lots of useful stuff here
}

What would be nice in a tuple is something like a forEach() method that iterates over all attributes:

tuple(1, "a", null).forEach(System.out::println);

The above would simply yield:

1
a
null

Now, what would this forEach() method’s argument type be? It would have to look like this:

class Tuple3<T1, T2, T3> {
    void forEach(Consumer<? super T1 | T2 | T3> c) {}
}

The consumer would receive an object that is of type T1 or T2 or T3. But a consumer that accepts a common super type of the previous three types is OK as well. For example, if we have:

Tuple2<Integer, Long> tuple = tuple(1, 2L);
tuple.forEach(v -> 
    System.out.println(v.doubleValue()));

The above would compile, because Number (or, more formally, Number & Comparable<?> is a common super type of Integer and Long, and it contains a doubleValue() method.

As soon as we’re adding e.g. a String to the tuple, the following will no longer compile:

Tuple3<Integer, Long, String> tuple = 
    tuple(1, 2L, "A");

// Doesn't compile
tuple.forEach((Number v) -> 
    System.out.println(v.doubleValue()));

Unfortunately, this is not possible in Java

Java currently supports union types (see also algebraic data types) only for exception catch blocks, where you can write things like:

interface X {
    default void print() {}
}
class X1 extends RuntimeException implements X {}
class X2 extends RuntimeException implements X {}

// With the above
try {
    ...
}
catch (X1 | X2 e) {
    // This compiles for the same reasons!
    e.print();
}

But unfortunately, catch blocks are the only place in Java that allows for using union types.

This is where Daniel’s clever and cunning workaround comes into play. We can write a static method that performs some “pattern-matching” (if you squint) using generics, the other way round:

static <
    T, 
    T1 extends T, 
    T2 extends T, 
    T3 extends T
> 
void forEach(
    Tuple3<T1, T2, T3> tuple, 
    Consumer<? super T> consumer
) {
    consumer.accept(tuple.v1);
    consumer.accept(tuple.v2);
    consumer.accept(tuple.v3);
}

The above can now be used typesafely to infer the common super type(s) of T1, T2, and T3:

Tuple2<Integer, Long> t = tuple(1, 2L);
forEach(t, c -> {
    System.out.println(c.doubleValue());
});

yielding, as expected:

1.0
2.0

It makes sense, because the generic type constraints are simply specified “the other way round”, i.e. when T1 extends T, forcibly, T super T1

If you squint really hard 😉

This technique is supposedly used by Daniel in Javaslang’s upcoming pattern matching API. We’re looking forward to seeing that in action!

Did you like this article?

Read also the following ones:

jOOQ Tuesdays: Glenn Paulley Gives Insight into SQL’s History


Welcome to the jOOQ Tuesdays series. In this series, we’ll publish an article on the third Tuesday every other month where we interview someone we find exciting in our industry from a jOOQ perspective. This includes people who work with SQL, Java, Open Source, and a variety of other related topics.

glenn-paulley

I’m very excited to feature today Glenn Paulley who has been working with and on SQL for several decades.

Glenn, you have been a part of the database ecosystem since the very early days, having been Director of Engineering at Sybase and representing SAP with the SQL Standard committee. What is it that fascinates you so much about databases?

Data management technology has been my favourite subject within Computer Science since I was an undergraduate student at the University of Manitoba, and so it has remained throughout much of my career. I was privileged to be part of the team that implemented the first online IBM DB2 application – using IMS/TM as the transaction monitor – in Canada at Great-West Life Assurance in the late 1980’s, and we hosted some of the DB2 development team from IBM’s San Jose lab – including Don Haderle – to celebrate that achievement. So yes, I guess you could say that I’ve been around for a while.

Much of my personal expertise lies in the realm of query processing and optimization, having had Paul Larson and Frank Wm. Tompa as my Ph.D. supervisors at the University of Waterloo. But I am interested in many related database subjects: query languages certainly, multidatabase systems, and information retrieval. Two closely-related topics within database systems are of particular interest to me.  One is scale. Companies and organizations have a lot of data; billion-row tables in a relational database are fairly routine today in many companies. Advances in other technologies, such as the Internet of Things, are going to dramatically increase the data management requirements for firms wanting to take advantage of these technologies. Implementing solutions to achieve the scale required is difficult, and is of great interest to me. The related issue, quite naturally, is processing queries over such vast collections and getting the execution time down to something reasonable. So query optimization remains a favourite topic.

We’ve had a very interesting E-Mail conversation about SQL’s experiments related to Object Orientation in the late 1990’s. Today, we have ORDBMS like Oracle, Informix, PostgreSQL. Why didn’t ORDBMS succeed?

ORDBMS implementations were not as successful as their developers expected, to be sure, though I would note that many SQL implementations now contain “object” features even though objects are specifically omitted from the ISO SQL standard. Oracle 12c is a good example – Oracle’s PL/SQL object implementation supports a good selection of object-oriented programming features, such as polymorphism and single inheritance, that when coupled with collection types provide a very rich data model that can handle very complex applications. There are others, too, of course: InterSystems’ Caché product, for example, is still available.

So, while object support in relational systems is present, in many instances, to me the significant issues are (1) the performance of object constructions on larger database instances and (2) the question of where do you want objects to exist in the application stack: in the server proper, or in another tier, implemented in a true object-oriented language? The latter issue is the premise behind object-relational mapping tools, though I think that their usage often causes as many problems as they solve.

How do you see the future of the SQL language – e.g. with respect to Big Data or alternative models like document stores (which have N1QL) or graph stores (which have Open Cypher)?

My personal view is that SQL will continue to evolve; having an independent query language that permits one to query or manipulate a database but avoid writing a “program” in the traditional sense is a good idea. Over time that language might evolve to something different from today’s SQL, but it is likely that we will still call it S-Q-L. I do not expect a revolutionary approach to be successful; there is simply far too much invested in current applications and infrastructure. Don’t get me wrong – I’m not saying SQL doesn’t need improvements. It does.

Unlike the work of e.g. the JCP or w3c, which are public and open, SQL still seems to be a more closed environment. How do you see the interaction between SQL and the end user? Can “ordinary” folks participate in the future of SQL?

The SQL standard is published by the International Standards Organization (ISO), whose member countries contribute to changes to the standard and vote on them on a regular basis. Member countries that contribute to the ISO SQL standard include (at least) the US, Canada, Germany, the United Kingdom, Japan, and Korea. Participation in the standards process requires individuals or companies to belong to these “national bodies” in order to view drafts of the standard and vote on proposed changes. Usually those meetings are held in-person – at least in Canada and the United States – so there is a real cost to participation in the process.

While having SQL as an international ISO standard is, I think, worthwhile, the ISO’s business model is based on activities such as collecting revenue from the sale of standards documents. Unless governments or other benefactors sponsor the development of those standards, then it is difficult to see how the standard can be made more freely available. That issue is a political one, rather than a technical one.

After a brief stop at Conestoga College, you’re heading back to SAP. What will you be working on in the next few years?

I will be continuing to focus on database technology now that I’ve returned to SAP. So far I’ve had a fantastic experience being back at the SAP Waterloo lab, and I have every expectation that will continue into the future.

You’ve spent a lot of time at SAP (and before SAP) with Sybase. How do the different Sybase distributions relate to SAP’s new flagship database HANA?

The various SAP database systems (SQL Anywhere, IQ, ASE) all contain database technology pertinent to the HANA platform.

Last question: What do SQL and Curling have in common? 🙂

As a Level 3 curling coach, one of my tasks as a High Performance Coach for the Ontario Curling Council is to collect performance data on the athletes in our various programs. Naturally I use a database system to store that data, and perform various analyses on it, so the two are not as unrelated as you might think!

 

Dear API Designer. Are You Sure, You Want to Return a Primitive?


Some APIs are set in stone. For instance, the JDK’s. Or public APIs, like the one between a database and a database client (e.g. JDBC).

This makes designing such APIs rather difficult as a lot of thinking needs to be done prior to publishing an API. Which means that being defensive when designing the API is a good choice.

One defensive API design strategy is to always work with parameter objects and return objects. We’ve already blogged about parameter objects before. Let’s have a look at an API, that doesn’t use return objects, and why that’s so terrible:

Database updatable statements

When fetching data from a database, we get back a convenient API type, the JDBC ResultSet. Other languages than Java have similar types to model database results. While the ResultSet mainly models a set of tuples, it also contains various additional useful features, like ResultSet.getMetaData() or ResultSet.getWarnings(), which are clever back-doors for passing arbitrary, additional information with the ResultSet.

What’s best about these result types is that they can be extended backwards-compatibly. New methods and features can be added to these result types, without modifying:

  • Any existing contracts
  • Any existing client code

The only thing that might break is JDBC drivers, but since Java 8, JDBC 4.2, and default methods, this is a thing of the past as well.

Things look quite different when calling an update statement in the database:

int count = stmt.executeUpdate();

Egh.

A count value. That’s it? What about any trigger-generated information? What about warnings (I know, they’re available from the statement. Which was modified by the call)?

Interestingly enough, this count value being an int seems to have bothered some people long enough for the method to have been de-facto overloaded in JDBC 4.2:

long count = stmt.executeLargeUpdate();

Hmm…

I’m saying “de-facto overloaded”, because it is really technically an overload, but because Java doesn’t support overloading by return type, the name was changed as well. (Well, the JVM does support it, but not the language).

When you read the Javadoc of executeUpdate() method, you will notice that different states are encoded in this single primitive value:

Returns: either (1) the row count for SQL Data Manipulation Language (DML) statements or (2) 0 for SQL statements that return nothing

What’s more, there’s a similar method called getUpdateCount(), which encodes even more complex state into a single primitive:

the current result as an update count; -1 if the current result is a ResultSet object or there are no more results

Egh…

And as if this wasn’t bad enough, here’s a very peculiar workaround for the above limitation was implemented by the MySQL database, which encodes different states for UPSERT statements as such:

With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if the row is inserted as a new row and 2 if an existing row is updated. – See here

If performance doesn’t matter, always return a reference type!

This is really bad. The call runs over the wire against a database. It is inherently slow. We wouldn’t lose anything if we had an UpdateResult data type as a result of executeUpdate(). A different example is String.indexOf(...) which encodes “not found” as -1 for performance reasons.

The mistake doesn’t only happen in these old APIs that pre-date object oriented programming. It is repeated again in newer APIs in many applications, when the first thing that comes to mind as being a useful method result is a primitive value (or worse: void).

If you’re writing a fluent API (like the Java 8 Stream API, or jOOQ), this will not be an issue as the API always returns the type itself, in order to allow for users to chain method calls.

In other situations, the return type is very clear, because you’re not implementing any side-effectful operation. But if you do, please, think again whether you really want to return just a primitive. If you have to maintain the API for a long time, you might just regret it some years later.

Read on about API design

The 5 Golden Rules of Giving Awesome Customer Support


This is usually a tech blog, but every now and then, we make an exception when there’s something important to say.

Today, I’m going to criticise a lot of our industry’s understanding of support.

Who is this article for?

It’s for every software engineer giving support to users and/or customers, and every manager who works with software engineers. In fact, it isn’t restricted to software, but since this is a Java blog, let’s pretend it is only about software.

#1: All users are customers

While we’re at it, let’s use the terms “users” and “customers” as synonyms, because every user should be treated like a paying customer. In fact, every user should be treated like the only paying customer you have, on which your whole business depends. This is why I will put “customer” in quotes in this article.

This is something you should constantly remind yourself of. It doesn’t matter if your “customer” paid you millions for your software, or if they just use your quick and dirty Open Source library that you implemented in your free time.

Why is that? Because it’s your fault and problem as a vendor if you chose some business model (or if you don’t even have a business model) that generates (excessive?) support work. Change the model. Make more non-paying “customers” pay for support. Help them understand the value they’re getting from being your paying customers. Or keep doing stuff for free, it doesn’t matter as long as you will remember that it’s your problem, not theirs.

Always take Apple as a shining example. Apple customers love to pay excessive amounts for average hardware and software with nice-looking design. This has nothing to do with support, it just shows that Apple has gotten making customers pay right. Because Apple users want to be paying customers.

#2: Your customer took the time

Remember this:

Your “customer” took the time to report an issue.

Period. That’s all there is to say. When a “customer” gets in touch with you, they took the time to get in touch. So you should be thankful they took the time. Not all “customers” take the time. Most of them just rant and leave you silently if something goes wrong. Or worse, they rant in public and then leave you.

If a “customer” does take the time, this means they stand for thousands of others who don’t take the time but like your product and want to continue liking it. Don’t spoil that. Keep those customers!

#3: Your customer is abrasive and demanding

The food chain goes this way round. Accept that. Moreover, remember: You’re not some divine entity that grants an audience to an unworthy soul who should kneel before thy wisdom. You’re the supplier. They’re the customer. They’re the king. You’re the servant.

Of course they are abrasive and demanding. They are the “customer”. They had a bad day. They have an idiot boss. Their job sucks. They work on legacy code. They blame you (and everyone else). It’s not their fault. It never was.

So what? That happens from time to time. More often than not, however, “customers” are nice people, and again: They took the time to get in touch.

So, your job is to thank them for that. To make sure you do everything needed to help them. To solve their trouble and issues. Which often isn’t what they’re saying. They just want you to listen. A workaround might just be enough. They’ll understand you can’t fix things immediately. But do help them.

#4: Your customer doesn’t understand your product

You’ve spent all that time with your product. You know exactly how it works. You have accepted all the caveats. The bugs. The workarounds.

Your customer doesn’t. They have more important things to do than reverse engineer your funky product. If they don’t understand something, always assume it’s your fault. You could improve the product. Or the documentation. Or your SEO to make sure they will find the documentation. Or your support channels (see below). Whatever. Always assume that your “customer” doesn’t have the time to really understand your product. Why should they.

Learn from your “customers”. Their misunderstanding of your ways is your chance to do it better. Every single time.

#5: Your customer doesn’t care about your workflows

And here’s the one thing you must never, NEVER do:

NEVER tell your “customer” that they’ve used the wrong support channel

NEVER do the above. They’ll learn by themselves after a while. Or they don’t. Does it matter? It doesn’t. They’ve taken the time to reach out and that ought to be enough.

Perhaps, you’ve chosen a confusing setup of support channels. Or complicated products (looking at you, Bugzilla). Improve it, then. Make it easier for your “customers” to choose the right support channel if they have gotten it “wrong”.

What does wrong support channel even mean? Wrong support channels create administrative work to you. You have to copy a bug report from Google Groups to JIRA. From your E-Mail program to GitHub. Etc.

So, a “customer” has reported a bug on the discussion group. You prefer they create an issue in an issue tracker. How do you react?

  • Bad: This is a bug not a discussion, we have a bug tracker for this. Can you please create an issue??!?
  • Good: Thank you very much for bringing this to our attention. I have registered an issue #717863 for this problem. We’re looking into this as quickly as possible and let you know about our progress. If there’s anything … blah blah.

As simple as that.

Conclusion

There are many more things that help you get support right. And it isn’t always easy. Sometimes, it’s just too hard to get it right. We all have tons of things to do. And priorities.

But remember one thing. This is (almost) never the “customer”‘s fault. And even if it is, you should never make them feel as if it were. There are enough enterprise-y companies out there whose processes are too complex and heavy to ever give awesome support.

Don’t be like those companies. Be awesome for your “customers”.