Java 8 Friday: Most Internal DSLs are Outdated

Posted on by lukaseder
At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Most Internal DSLs are Outdated

That’s quite a statement from a vendor of one of the most advanced internal DSLs currently on the market. Let me explain:

Languages are hard

Learning a new language (or API) is hard. You have to understand all the keywords, the constructs, the statement and expression types, etc. This is true both for external DSLs, internal DSLs and “regular” APIs, which are essentially internal DSLs with less fluency. When using JUnit, people have grown used to using hamcrest matchers. The fact that they’re available in six languages (Java, Python, Ruby, Objective-C, PHP, Erlang) makes them somewhat of a sound choice. As a domain-specific language, they have established idioms that are easy to read, e.g. 

assertThat(theBiscuit, equalTo(myBiscuit));
assertThat(theBiscuit, is(equalTo(myBiscuit)));
assertThat(theBiscuit, is(myBiscuit));
When you read this code, you will immediately “understand” what is being asserted, because the API reads like prosa. But learning to write code in this API is harder. You will have to understand:
  • Where all of these methods are coming from
  • What sorts of methods exist
  • Who might have extended hamcrest with custom Matchers
  • What are best practices when extending the DSL
For instance, in the above example, what exactly is the difference between the three? When should I use one and when the other? Is is() checking for object identity? Is equalTo() checking for object equality? The hamcrest tutorial goes on with examples like these: 

public void testSquareRootOfMinusOneIsNotANumber() {
    assertThat(Math.sqrt(-1), is(notANumber()));
}
You can see that notANumber() apparently is a custom matcher, implemented some place in a utility: 

public class IsNotANumber
extends TypeSafeMatcher<Double> {

  @Override
  public boolean matchesSafely(Double number) {
    return number.isNaN();
  }

  public void describeTo(Description description) {
    description.appendText("not a number");
  }

  @Factory
  public static <T> Matcher<Double> notANumber() {
    return new IsNotANumber();
  }
}
While this sort of DSL is very easy to create, and probably also a bit fun, it is dangerous to start delving into writing and enhancing custom DSLs for a simple reason. They’re in no way better than their general-purpose, functional counterparts – but they’re harder to maintain. Consider the above examples in Java 8:

Replacing DSLs with Functions

Let’s assume we have a very simple testing API: 

static <T> void assertThat(
    T actual, 
    Predicate<T> expected
) {
    assertThat(actual, expected, "Test failed");
}

static <T> void assertThat(
    T actual, 
    Predicate<T> expected, 
    String message
) {
    assertThat(() -> actual, expected, message);
}

static <T> void assertThat(
    Supplier<T> actual, 
    Predicate<T> expected
) {
    assertThat(actual, expected, "Test failed");
}

static <T> void assertThat(
    Supplier<T> actual, 
    Predicate<T> expected, 
    String message
) {
    if (!expected.test(actual.get()))
        throw new AssertionError(message);
}
Now, compare the hamcrest matcher expressions with their functional equivalents: 

// BEFORE
// ---------------------------------------------
assertThat(theBiscuit, equalTo(myBiscuit));
assertThat(theBiscuit, is(equalTo(myBiscuit)));
assertThat(theBiscuit, is(myBiscuit));

assertThat(Math.sqrt(-1), is(notANumber()));

// AFTER
// ---------------------------------------------
assertThat(theBiscuit, b -> b == myBiscuit);
assertThat(Math.sqrt(-1), n -> Double.isNaN(n));
With lambda expressions, and a well-designed assertThat() API, I’m pretty sure that you won’t be looking for the right way to express your assertions with matchers any longer. Note that unfortunately, we cannot use the Double::isNaN method reference, as that would not be compatible with Predicate<Double>. For that, we’d have to do some primitive type magic in the assertion API, e.g. 

static void assertThat(
    double actual, 
    DoublePredicate expected
) { ... }
Which can then be used as such: 

assertThat(Math.sqrt(-1), Double::isNaN);

Yeah, but…

… you may hear yourself saying, “but we can combine matchers with lambdas and streams”. Yes, of course we can. I’ve just done so now in the jOOQ integration tests. I want to skip the integration tests for all SQL dialects that are not in a list of dialects supplied as a system property: 

String dialectString = 
    System.getProperty("org.jooq.test-dialects");

// The string must not be "empty"
assumeThat(dialectString, not(isOneOf("", null)));

// And we check if the current dialect() is
// contained in a comma or semi-colon separated
// list of allowed dialects, trimmed and lowercased
assumeThat(
    dialect().name().toLowerCase(),

    // Another matcher here
    isOneOf(stream(dialectString.split("[,;]"))
        .map(String::trim)
        .map(String::toLowerCase)
        .toArray(String[]::new))
);
… and that’s pretty neat, too, right? But why don’t I just simply write: 

// Using Apache Commons, here
assumeThat(dialectString, StringUtils::isNotEmpty);
assumeThat(
    dialect().name().toLowerCase(),
    d -> stream(dialectString.split("[,;]"))
        .map(String::trim)
        .map(String::toLowerCase())
        .anyMatch(d::equals)
);
No Hamcrest needed, just plain old lambdas and streams! Now, readability is a matter of taste, of course. But the above example clearly shows that there is no longer any need for Hamcrest matchers and for the Hamcrest DSL. Given that within the next 2-3 years, the majority of all Java developers will be very used to using the Streams API in every day work, but not very used to using the Hamcrest API, I urge you, JUnit maintainers, to deprecate the use of Hamcrest in favour of Java 8 APIs.

Is Hamcrest now considered bad?

Well, it has served its purpose in the past, and people have grown somewhat used to it. But as we’ve already pointed out in a previous post about Java 8 and JUnit Exception matching, yes, we do believe that we Java folks have been barking up the wrong tree in the last 10 years. The lack of lambda expressions has lead to a variety of completely bloated and now also slightly useless libraries. Many internal DSLs or annotation-magicians are also affected. Not because they’re no longer solving the problems they used to, but because they’re not Java-8-ready. Hamcrest’s Matcher type is not a functional interface, although it would be quite easy to transform it into one. In fact, Hamcrest’s CustomMatcher logic should be pulled up to the Matcher interface, into default methods. Things dont’ get better with alternatives, like AssertJ, which create an alternative DSL that is now rendered obsolete (in terms of call-site code verbosity) through lambdas and the Streams API. If you insist on using a DSL for testing, then probably Spock would be a far better choice anyway.

Other examples

Hamcrest is just one example of such a DSL. This article has shown how it can be almost completely removed from your stack by using standard JDK 8 constructs and a couple of utility methods, which you might have in JUnit some time soon, anyway. Java 8 will bring a lot of new traction into last decade’s DSL debate, as also the Streams API will greatly improve the way we look at transforming or building data. But many current DSLs are not ready for Java 8, and have not been designed in a functional way. They have too many keywords for things and concepts that are hard to learn, and that would be better modelled using functions. An exception to this rule are DSLs like jOOQ or jRTF, which are modelling actual pre-existing external DSLs in a 1:1 fashion, inheriting all the existing keywords and syntax elements, which makes them much easier to learn in the first place.

What’s your take?

What is your take on the above assumptions? What is your favourite internal DSL, that might vanish or that might be completely transformed in the next five years because it has been obsoleted by Java 8? Stay tuned for more Java 8 Friday articles here on this blog.

Published by lukaseder

I made jOOQ

8 thoughts on “Java 8 Friday: Most Internal DSLs are Outdated”

  1. The major thing that you’d be throwing out with Hamcrest is the reporting of why things don’t match. In tests that’s a lot of the battle.

    Reply

    1. It is quite likely that 80% of hamcrest can be deleted once hamcrest is “lambda-i-fied”, as it wouldn’t be idiomatic Java 8… That’s just a bold claim without evidence, of course.

      Reply

  3. Great article! But I must disagree with your opinion. :) Here’s why: as others have stated the main usage to Hamcrest is its reusable components and its error reporting. I have written many matchers for my team that take complex objects and simplifies their assertion. For example:

    assertThat(theBiscuit, matchesBiscuit(myBiscuit));

    Here we assume that “theBiscuit” and “myBiscuit” both implement the interface “Biscuit”. Thus the BiscuitMatcher knows about Biscuits and how to determine if they are equivalent. Furthermore, the BiscuitMatcher knows about other matchers to match individual properties of Biscuits.

    In your example:

    assertThat(theBiscuit, is(myBiscuit));

    one has to make the big assumption that the implementation of theBiscuit implements the equals method. Furthermore in your lambda example:

    assertThat(theBiscuit, b -> b == myBiscuit);

    It would only work if theBiscuit and myBiscuit are the exact same instance.

    All caveats aside, I would find it difficult to believe that the lambda version that you presented is more legible than the version I present above. And at the end of the day, matchers are not hurting anything while providing legible documentation for method behaviors.

    Reply

    1. … It would only work if theBiscuit and myBiscuit are the exact same instance.

      OK, we can engage in the == vs. equals() discussion, which is a very old one in the Java ecosystem, but that’s not the point here.

      The point is that lambdas will become ubiquitous – if they aren’t already, and thus legibility will bias more strongly towards lambda expressions and the use of the Stream API (and other functionally enhanced, general-purpose collections API). I’m saying these things from the standpoint of someone who has maintained medium size systems with tons of “clever” API that was implemented to be “legible”, in which I could easily replace 100s of lines with 2-3 lines by simply replacing “legibility” by “elegance”. From a maintenance perspective, this has always turned a lot of things more reusable, and less error-prone for me.

      Of course, it’s hard to make a point here, when the examples are as silly as comparing biscuits :-)

      Reply

      1. I didn’t mention == to discuss == vs .equals because I wanted to discuss the difference between the two (I assume that anyone who cares about this topic would know the difference), rather I mentioned it to point out that quality matchers cannot be replaced by == or .equals.

        Essentially, one can not just take a well written matcher and replace it with assertThat(theBiscuit, b -> b == myBiscuit); or assertThat(theBiscuit, b -> Objects.equals(b, myBiscuit)); Both scenarios make big assumptions that are often times not true. For example, theBiscuit might be a PojoBiscuit and myBiscuit might be a Mockito mocked Biscuit. In that case, neither option will work.

        But let’s say for a second that I give the developers the benefit of the doubt and assume that everyone has implemented .equals and have done so in a way that objects that are different implementations of the same interface will come up true if their properties match. Then while testing new functionality, it turns out that one biscuit has more milk in it than the other. The assertion will fail, but you will not know why without doing some debugging. You’ll just get an error back that says the biscuits don’t match. Wouldn’t it be nicer to know right away that they didn’t match because one has more milk in it than expected?

        The point I’m driving at isn’t that everyone must be using Hamcrest or any other specific testing framework, but rather that I don’t agree that lambdas and function references render things like Hamcrest as obsolete. I love lambdas, have used it to enhance my own tests, and will even be giving a conference talk about the joys of Java 8; but there’s still a place for reusable test components that report reasons for failures.

        Reply

        1. The article was written in a black/white tone, yes. The point here, however, is that a Hamcrest Matcher is not a functional interface, which generates (due to the nature of the Java language) a lot of boilerplate. Things that can be avoided by writing utilities that embrace functional programming. If you can achieve this with matchers, fine! But I sincerely hope that with new language tools, developers will stop writing 20 lines of code to express an individual semantic that can be written (and composed) in a one-liner function.

          I actually believe we do agree, if only we didn’t get hung up on the equals() non-discussion ;-)

          Reply

Leave a Reply