Defensive API evolution with Java interfaces

Posted on by lukaseder
API evolution is something absolutely non-trivial. Something that only few have to deal with. Most of us work on internal, proprietary APIs every day. Modern IDEs ship with awesome tooling to factor out, rename, pull up, push down, indirect, delegate, infer, generalise our code artefacts. These tools make refactoring our internal APIs a piece of cake. But some of us work on public APIs, where the rules change drastically. Public APIs, if done properly, are versioned. Every change – compatible or incompatible – should be published in a new API version. Most people will agree that API evolution should be done in major and minor releases, similar to what is specified in semantic versioning. In short: Incompatible API changes are published in major releases (1.0, 2.0, 3.0), whereas compatible API changes / enhancements are published in minor releases (1.0, 1.1, 1.2). If you’re planning ahead, you’re going to foresee most of your incompatible changes a long time before actually publishing the next major release. A good tool in Java to announce such a change early is deprecation.

Interface API evolution

Now, deprecation is a good tool to indicate that you’re about to remove a type or member from your API. What if you’re going to add a method, or a type to an interface’s type hierarchy? This means that all client code implementing your interface will break – at least as long as Java 8’s defender methods aren’t introduced yet. There are several techniques to circumvent / work around this problem:

1. Don’t care about it

Yes, that’s an option too. Your API is public, but maybe not so much used. Let’s face it: Not all of us work on the JDK / Eclipse / Apache / etc codebases. If you’re friendly, you’re at least going to wait for a major release to introduce new methods. But you can break the rules of semantic versioning if you really have to – if you can deal with the consequences of getting a mob of angry users. Note, though, that other platforms aren’t as backwards-compatible as the Java universe (often by language design, or by language complexity). E.g. with Scala’s various ways of declaring things as implicit, your API can’t always be perfect.

2. Do it the Java way

The “Java” way is not to evolve interfaces at all. Most API types in the JDK have been the way they are today forever. Of course, this makes APIs feel quite “dinosaury” and adds a lot of redundancy between various similar types, such as StringBuffer and StringBuilder, or Hashtable and HashMap. Note that some parts of Java don’t adhere to the “Java” way. Most specifically, this is the case for the JDBC API, which evolves according to the rules of section #1: “Don’t care about it”.

3. Do it the Eclipse way

Eclipse’s internals contain huge APIs. There are a lot of guidelines how to evolve your own APIs (i.e. public parts of your plugin), when developing for / within Eclipse. One example about how the Eclipse guys extend interfaces is the IAnnotationHover type. By Javadoc contract, it allows implementations to also implement IAnnotationHoverExtension and IAnnotationHoverExtension2. Obviously, in the long run, such an evolved API is quite hard to maintain, test, and document, and ultimately, hard to use! (consider ICompletionProposal and its 6 (!) extension types)

4. Wait for Java 8

In Java 8, you will be able to make use of defender methods. This means that you can provide a sensible default implementation for your new interface methods as can be seen in Java 1.8’s java.util.Iterator (an extract): 

public interface Iterator<E> {

    // These methods are kept the same:
    boolean hasNext();
    E next();

    // This method is now made "optional" (finally!)
    public default void remove() {
        throw new UnsupportedOperationException("remove");
    }

    // This method has been added compatibly in Java 1.8
    default void forEach(Consumer<? super E> consumer) {
        Objects.requireNonNull(consumer);
        while (hasNext())
            consumer.accept(next());
    }
}
Of course, you don’t always want to provide a default implementation. Often, your interface is a contract that has to be implemented entirely by client code.

5. Provide public default implementations

In many cases, it is wise to tell the client code that they may implement an interface at their own risk (due to API evolution), and they should better extend a supplied abstract or default implementation, instead. A good example for this is java.util.List, which can be a pain to implement correctly. For simple, not performance-critical custom lists, most users probably choose to extend java.util.AbstractList instead. The only methods left to implement are then get(int) and size(), The behaviour of all other methods can be derived from these two: 

class EmptyList<E> extends AbstractList<E> {
    @Override
    public E get(int index) {
        throw new IndexOutOfBoundsException("No elements here");
    }

    @Override
    public int size() {
        return 0;
    }
}
A good convention to follow is to name your default implementation AbstractXXX if it is abstract, or DefaultXXX if it is concrete

6. Make your API very hard to implement

Now, this isn’t really a good technique, but just a probable fact. If your API is very hard to implement (you have 100s of methods in an interface), then users are probably not going to do it. Note: probably. Never underestimate the crazy user. An example of this is jOOQ’s org.jooq.Field type, which represents a database field / column. In fact, this type is part of jOOQ’s internal domain specific language, offering all sorts of operations and functions that can be performed upon a database column. Of course, having so many methods is an exception and – if you’re not designing a DSL – is probably a sign of a bad overall design.

7. Add compiler and IDE tricks

Last but not least, there are some nifty tricks that you can apply to your API, to help people understand what they ought to do in order to correctly implement your interface-based API. Here’s a tough example, that slaps the API designer’s intention straight into your face. Consider this extract of the org.hamcrest.Matcher API: 

public interface Matcher<T> extends SelfDescribing {

    // This is what a Matcher really does.
    boolean matches(Object item);
    void describeMismatch(Object item, Description mismatchDescription);

    // Now check out this method here:

    /**
     * This method simply acts a friendly reminder not to implement 
     * Matcher directly and instead extend BaseMatcher. It's easy to 
     * ignore JavaDoc, but a bit harder to ignore compile errors .
     *
     * @see Matcher for reasons why.
     * @see BaseMatcher
     * @deprecated to make
     */
    @Deprecated
    void _dont_implement_Matcher___instead_extend_BaseMatcher_();
}
“Friendly reminder”, come on. ;-)

Other ways

I’m sure there are dozens of other ways to evolve an interface-based API. I’m curious to hear your thoughts!

Published by lukaseder

I made jOOQ

9 thoughts on “Defensive API evolution with Java interfaces”

  1. I personally like and do the “Abstract” method method (intended). Interfaces in their current form (< Java 8) in my opinion should only be used for API and to emulate multiple inheritance.

    I have worked with Enterprise Java software most of my career both internal and external and the sheer plethora of lets make tons of "Interfaces because it looks like good design" is painful.

    Unless your writing a framework and not a library, application, etc… you should rarely need to use an interface when an abstract class will work. Even modern AOP does not require proxy interfaces.

    Even for frameworks I would not make interfaces till a developer complains about the missing functionality.

    The other reason why I feel interfaces are overrated is at least for me is I have been doing more and more message passing architecture (message bus). Thus the message (immutable POJO) is the interface. I think we are going to continue to see more architectures like this given the async needs of today (see Vert.x and Akka).

    I don't know if jOOQ had the multiple inheritance problem but I did feel like there were just boundless interfaces. Maybe Java 8 will make the code tighter?

    Also I think you missed two important techniques for library evolution:

    * Package renaming/mangling (ie commons lang3)
    * And bridge method injector: http://kohsuke.org/2010/08/07/potd-bridge-method-injector/

    I do enjoy reading your blog! It motivates me to blog more.

    Reply

    1. Thanks for your thoughts! Looking forward to your own posts!

      Interfaces are essential to jOOQ as its DSL depends on Java interfaces’ multiple inheritance capability. See this post here:

      https://blog.jooq.org/2012/01/05/the-java-fluent-api-designer-crash-course/

      With Java 8’s interfaces closing in on what Scala calls “traits”, we’re somewhat getting where we should be in my opinion – although I really do despise the new, useless “default” keyword. I’ve tried my luck on the lambda-dev mailing list, without success :-)

      https://mail.openjdk.java.net/pipermail/lambda-dev/2012-August/005393.html

      I guess this multiple inheritance business will always give grounds to heated discussions – ultimately as it boils down to the composition over inheritance question. I.e. when an object implements 3 interfaces, shouldn’t it rather have (and give access to) 3 properties?

      I like your point about messaging. Essentially, this point isn’t really about messaging, but about serialisation of state. Interfaces do not encapsulate any state and are thus the wrong thing to use when it comes to messaging. But nothing keeps you from letting your messages implement interfaces in order to adhere to specific APIs

      Reply

        1. I see… Well I’m looking forward to Java 8, which will clean things up a little. These reflective, “aspective”, and other Java enhancements are kind of weird for someone who heavily relies on their IDE to tell them the truth about their codebase. E.g. how can you statically analyse call graphs with AOP the way you showed in your Stack Overflow answer?

          Reply

          1. You can analyze the code with the proper plugins. The Eclipse AspectJ plugin does a very good job of this showing where the advice is coming from. Even the exceptions and the debugger work correctly. The downside is that its another plugin to bloat your Eclipse install and complicate your build but both the Maven plugin and Eclipse plugin are very very mature…. but it does have static analysis.

            Compare this to Ruby and Groovy in the JVM where all kinds of magic is being applied that you can’t analyze and Scala where the IDE tools are still not to the level of the JDT and AspectJ plugins …. I think AspectJ is the under-appreciated super-powerful lesser evil.

            Also AspectJ does things that Scala cannot like apply advice across methods based on name and variable arguments. AspectJ does not work with Scala. But it does work with the Xtend language. IMHO AspectJ + Xtend is the killer combination giving you meta-programming that is only possible with something like Groovy/Ruby but with the conciseness, type safety and speed of Scala.

            The only thing that Groovy and Scala have that Java still doesn’t and won’t have is some sort of LWP (light weight process) concurrency. Groovy has Gpars and Scala has Actors… envious I am.

            Reply

Leave a Reply