Beware of Functional Programming in Java!


This isn’t going to be a rant about functional programming, which is awesome. This is a warning about some practices that you are very likely going to apply to your code, which are terribly wrong!.

Higher order functions are essential to functional programming, and thus, talking about them will help you be the center of attention at parties.

If you’re writing JavaScript, you’re doing it all the time. For instance:

setTimeout(function() {
    alert('10 Seconds passed');
}, 10000);

The above setTimeout() function is a higher-order function. It is a function that takes an anonymous function as an argument. After 10 seconds, it will call the function passed as an argument.

We can write another easy higher-order function that provides the above function as a result:

var message = function(text) {
    return function() {
        alert(text);
    }
};

setTimeout(message('10 Seconds passed'), 10000);

If you execute the above, message() will be executed, returning an anonymous function, which alerts the argument text you have passed to message()

In functional programming, the above is common practice. A function returned from a higher-order function will capture the outer scope and is able to act on this scope when called.

Why is this practice treacherous in Java?

For the same reasons. A “function” (lambda) returned from a higher-order “function” (method) will capture the outer scope and is able to act on this scope when called.

The most trivial example is given here:

class Test {
    public static void main(String[] args) {
        Runnable runnable = runnable();
        runnable.run(); // Breakpoint here
    }

    static Runnable runnable() {
        return () -> {
            System.out.println("Hello");
        };
    }
}

In the above logic, if you put a breakpoint right where the runnable.run() call is made, you can see the harmless lambda instance on the stack. A simple generated class, backing the functional interface implementation:

harmless-lambda-instance

Now let’s translate this example to your average Enterprise™ application (notice the annotations), which we’ve greatly simplified to fit this blog post:

class Test {
    public static void main(String[] args) {
        Runnable runnable = new EnterpriseBean()
            .runnable();
        runnable.run(); // Breakpoint here
    }
}

@ImportantDeclaration
@NoMoreXML({
    @CoolNewValidationStuff("Annotations"),
    @CoolNewValidationStuff("Rock")
})
class EnterpriseBean {
    Object[] enterpriseStateObject = 
        new Object[100_000_000];

    Runnable runnable() {
        return () -> {
            System.out.println("Hello");
        };
    }
}

The breakpoint is still at the same spot. What do we see on the stack?

Still a harmless little lambda instance:

harmless-lambda-instance-2

Fine. Of course. Let’s add some additional logging, just for debugging

class Test {
    public static void main(String[] args) {
        Runnable runnable = new EnterpriseBean()
            .runnable();
        runnable.run(); // Breakpoint here
    }
}

@ImportantDeclaration
@NoMoreXML({
    @CoolNewValidationStuff("Annotations"),
    @CoolNewValidationStuff("Rock")
})
class EnterpriseBean {
    Object[] enterpriseStateObject = 
        new Object[100_000_000];

    Runnable runnable() {
        return () -> {
            // Some harmless debugging here
            System.out.println("Hello from: " + this);
        };
    }
}

Ooops!

Suddenly, the “harmless” little this reference forced the Java compiler to enclose the enclosing instance of the EnterpriseBean™ in the returned Runnable class:

treacherous-lambda-with-enclosing-instance

And with it that heavy enterpriseStateObject came along, which can now no longer be garbage collected, until the call site releases the harmless little Runnable

Sure. Just be careful, know what you’re doing, and don’t reference “this” from a lambda

… you say?

How about a more subtle version?

class EnterpriseBean {
    Object[] enterpriseStateObject = 
        new Object[100_000_000];

    Runnable runnable() {
        return () -> log(); // implicit this.log()
    }

    void log() {
        // Some harmless debugging here
        System.out.println("Hello");
    }
}

OK, this is nothing new now, is it?

Indeed, it isn’t. Java 8 doesn’t have first-class functions, and that’s OK. The idea of backing lambda expressions by nominal SAM types is quite cunning, as it allowed to upgrade and lambda-y-fy all existing libraries in the Java ecosystem without changing them.

Also, with an anonymous class, this whole story would not have been surprising. The following coding style has leaked internal state via anonymous classes since good old Swing 1.0 style ActionListener et al.

class Test {
    public static void main(String[] args) {
        Runnable runnable = new EnterpriseBean()
            .runnable();
        runnable.run();
    }
}

@ImportantDeclaration
@NoMoreXML({
    @CoolNewValidationStuff("Annotations"),
    @CoolNewValidationStuff("Rock")
})
class EnterpriseBean {
    Object[] enterpriseStateObject = 
        new Object[100_000_000];

    Runnable runnable() {
        return new Runnable() {
            @Override
            public void run() {
                System.out.println("Hello from " + EnterpriseBean.this);
            }
        };
    }
}

What’s new? The lambda style will encourage using higher-order functions in Java, all over the place. Which is generally good. But only when the higher-order function is a static method, whose resulting types will not enclose any state.

With the above examples, however, we can see that we’re going to be debugging through a couple of memory leaks and problems in the near future, when we start embracing Java 8’s functional style programming.

So, be careful, and follow this rule:

(“Pure”) Higher order functions MUST be static methods in Java!

Further reading

Enclosing instances have caused issues before. Read about how the dreaded double curly braces anti pattern has caused pain and suffering among Java developers for the last two decades.

16 thoughts on “Beware of Functional Programming in Java!

  1. There’s one thing I don’t understand. If I have a static higher order function but one that takes a lambda or anonymous inner class that captures the “this” of the calling class, won’t I still have the same problem? That is, the first class function that I pass to the higher order function has that massive class in its scope.

    • Yes, that can still happen. However, in most cases, it seems less “risky” to pass a (scope-capturing) function you control to some library method, rather than to return a (scope-capturing) function from a library method to the universe. In the first case, you can assess the risk. In the second case, you cannot as you may not know who will consume your function.

  2. Cool post, thanks. Once you laid out the problem it’s pretty simple to understand, but I admit I didn’t think of it until I read your explanation.

  3. I’m used to Ruby, where this problem exists in spades… And can’t be gotten rid of with anything like “static”, alas :-/

    (Ruby allows you to extract a scope from a lambda, and then pull all the stuff the lambda doesn’t use out from that scope. So it can’t be made safe from this problem, full stop, except by never hanging onto your lambdas for long.)

    • Interesting, I didn’t know that. I suspect the same is true for JavaScript, though and many other languages that know first-class functions, but not top-level functions…?

  4. I wouldn’t say capturing ‘this’ is something wholly bad, as it follows the same single principle as elsewhere in Java lambda: capture value, not variable. If the captured value is the Object (i.e. its reference) of a local variable, then content of the object is changeable to anyone else that’s still keeping a reference. To make the capturing lambda fully ‘pure’, the captured Object has to be immutable, which is not always the case in Java land. So it’s just the creator of the lambda needs to keep in mind there could be shared states introduced.

    • It certainly isn’t wholly bad – but even when the shared state is immutable, you’ll be (potentially accidentally) hanging on to it for too long, such that the GC can’t collect it. Given that immutability as a strategy also highly depends on a powerful GC, it’s certainly something worth thinking about.

      • I don’t think GC is an issue here. The captured Object is there because you are still in need of the lambda, which is in need of the Object.

        It’s another question if you just need part of the captured Object, because now either the Object should be spitted into smaller models and thus takes less memory, or you should use extra local variable for the lambda to capture just fields instead of the whole object.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s