QueryDSL vs. jOOQ. Feature Completeness vs. Now More Than Ever

This week, Timo Westkämper from QueryDSL has announced feature completeness on the QueryDSL user group, along with his call for contributions and increased focus on bugfixes and documentation.

Timo and us, we have always been in close contact, observing each other’s products. In the beginning of jOOQ in 2009, QueryDSL was ahead of us.

But we learned quickly and removed all of our shortcomings such that jOOQ and QueryDSL were quickly at eye level by 2011. Ever since, we have been taking inspiration from one another, as in the end, we have had similar goals. Today, whenever someone is looking for a querying DSL, people tend to recommend either of our tools:

QueryDSL is often a good choice in JPA-based environments, while jOOQ is mostly the best choice in SQL-based environments, although jOOQ has already been given some credit in JPA-based environments as well:

Anyway, today, we’d like to congratulate Timo to his new job, and to QueryDSL’s feature completeness.

jOOQ, on the other hand, is far from feature complete.

jOOQ is what SQLJ should have been from the beginning.

We’re only at the beginning. Java and SQL are the two platforms that are used by most of the developers on this planet. According to db-engines.com, almost every popular DBMS is a SQL-based relational DBMS. According to TIOBE, Java currently ranks #2 among all languages.

We strongly believe that all of these developers are in dire need for better SQL integration into the Java language. While ORMs and JPA are very well integrated, SQL is not, and that is what we are working on. jOOQ will be feature complete when the Java compiler can natively compile actual SQL code and SQL code fragments into jOOQ, which will serve as its backing AST model for further SQL transformation.

Until we reach that goal, we’ll be adding support for more SQL goodness. A small selection of things that we already support, beyond QueryDSL’s “feature completeness”:

  • Table-valued functions
  • PIVOT tables
  • DDL (with jOOQ 3.4)
  • MERGE statement
  • Derived tables and derived column lists
  • Row value expressions
  • Flashback query
  • Window functions
  • Ordered aggregate functions
  • Common table expressions (with jOOQ 3.4)
  • Object-oriented PL/SQL
  • User-defined types
  • Hierarchical SQL
  • Custom SQL transformation
  • 16 supported RDBMS (even MS Access!)
  • … you name it

Our roadmap is full of great ideas. There’s plenty of work, so let’s get going! Join us, your partner for…

jOOQ is the best way to write SQL in Java

A SQL query DSL for Scala by ScalikeJDBC

There are a tremendous amount of SQL APIs natively written in Scala. Manuel Bernhardt has summarised a nice collection in his a post. Another collection of Scala SQL APIs can be seen in this Stack Overflow question.

One API that we want to focus on in particular is ScalikeJDBC (licensed ASL 2.0), which has recently published a SQL query DSL API similar to that of jOOQ. See the full documentation here:

http://scalikejdbc.org/documentation/query-dsl.html

A couple of examples:

val orders: List[Order] = withSQL {
  select
    .from(Order as o)
    .innerJoin(Product as p).on(o.productId, p.id)
    .leftJoin(Account as a).on(o.accountId, a.id)
    .where.eq(o.productId, 123)
    .orderBy(o.id).desc
    .limit(4)
    .offset(0)
  }.map(Order(o, p, a)).list.apply()

The above example looks very similar to jOOQ code, except that the SELECT DSL seems to be a bit more rigid than jOOQ’s. For instance, it is not immediately obvious how to connect several complex predicates in that WHERE clause, or if complex predicates are available at all.

What’s really nice, however, is their way of leveraging Scala language features to provide a very fluent way of constructing dynamic SQL, as can be seen in this example:

def findOrder(id: Long, accountRequired: Boolean) = 
withSQL {
  select
    .from[Order](Order as o)
    .innerJoin(Product as p).on(o.productId, p.id)
    .map { sql =>
      if (accountRequired) 
        sql.leftJoin(Account as a)
           .on(o.accountId, a.id)
      else 
        sql
    }.where.eq(o.id, 13)
  }.map { rs =>
    if (accountRequired) 
      Order(o, p, a)(rs) 
    else 
      Order(o, p)(rs)
  }.single.apply()

From how we understand things, the map method that is invoked in the middle of the SQL statement (between innerJoin and where) can transform the intermediate DSL state using a lambda expression that allows for appending a leftJoin if needed. Obviously, this can be done in a more procedural fashion as well, by assigning that intermediate DSL state to a local variable.

The need for SQL query DSLs

We’ve blogged about many of these similar SQL query DSLs in the past. The fact that they constantly pop up in various APIs is no coincidence. SQL is a very typesafe and composable language that is hard to use dynamically through string-based APIs such as JDBC, ODBC, etc.

Having a typesafe internal domain-specific language model SQL in a host language like Java or Scala brings great advantages. But the disadvantages may shine through quickly, when the DSL is not carefully crafted in a completely foreseeable way. Take the following ScalikeJDBC QueryDSL example, for instance:

val ids = withSQL {
  select(o.result.id).from(Order as o)
    .where(sqls.toAndConditionOpt(
      productId.map(id => sqls.eq(o.productId, id)),
      accountId.map(id => sqls.eq(o.accountId, id))
    ))
    .orderBy(o.id)
}.map(_.int(1)).list.apply()

This toAndConditionOpt method is really unexpected and doesn’t follow the principle of least astonishment.

This is why jOOQ’s API design is based on a formal BNF that closely mimicks SQL itself. Read more about that here.

OneWebSQL™: Another competitor in the SQL schema generation “business”

I’ve been observing OneWebSQL™ for a year now, impatiently expecting its first public version and official go-live. I first discovered it during my regular jOOQ marketing work. I’m always closely observing competition for several reasons:

  • It shows that the market is alive and active, which helps making jOOQ even more relevant
  • jOOQ can incorporate the best ideas of competitor products
  • It’s always fun to meet people who share similar opinions

All of the above is a big motivation to make jOOQ better. For more competitor comparisons, see also the following recent articles:

OneWebSQL™

OneWebSQL™ is one of the more interesting ones, from a professional perspective. Similar to QueryDSL, OneWebSQL™ is backed by a company who has developed a SQL code generation and execution framework over the past years. Unlike jOOQ and QueryDSL, however, OneWebSQL™ is neither free nor open source. So lets have a brief look at what OneWebSQL™ is and how it compares to jOOQ and QueryDSL

Comparing jOOQ, QueryDSL, OneWebSQL™

The below list of compared features and attributes is meant to be objective and unbiased, even if it was written by the developer of jOOQ. Here goes

License

jOOQ ASL 2.0, free and open source
QueryDSL ASL 2.0, free and open source
OneWebSQL™ Proprietary license, closed source and partially obfuscated. Free runtime, somewhat expensive source code generator

Vendor and community

jOOQ Independent developer, active community
QueryDSL Mysema, active community
OneWebSQL™ e-point, newcomer with undisclosed community

Scope

Feature jOOQ QueryDSL OneWebSQL™
Source code generation [1] yes yes yes
Active Records [2] yes partially
(through JPA)
partially
(through generated DAOs)
Procedure support [3] yes no partially
(some JDBC abstraction)
UDT support [3] yes no no
DSL [4] yes
(formal BNF notation)
partially
(chained methods)
partially
(chained methods)
DML support yes yes yes
DDL support no no no
Supported RDBMS 13 9
(or more, using JPA)
5
SQL dialect abstraction [5] yes yes partially
(simulates some syntax elements)
SQL dialect simulation [6] yes yes no
Scala examples [7] Partially
Proof of concept
Partially
Proof of concept
Probably (?)

Some explanations

  • [1]: jOOQ and QueryDSL can reverse-engineer database meta data directly from a database. OneWebSQL™ expects a proprietary model file
  • [2]: “Active Records” are record classes that can persist (store, refresh, delete) themselves or be passed to an EntityManager, DAO, etc for persistence.
  • [3]: Stored procedures (and especially functions) can be used directly in SQL
  • [4]: A domain-specific language is best defined using a BNF notation. This can be translated to Java interfaces as described in a recent blog post. Simpler fluent APIs can still be constructed using “chained methods”. Strictly speaking, that is not a domain-specific language, though
  • [5]: SQL dialects vary greatly in subtle ways. Any SQL building tool should be able to distinguish dialects. This blog contains many examples, such as RDBMS bind variable casting madness
  • [6]: Many syntax elements (and more specifically functions) can be simulated in dialects that do not natively support them. This blog contains many examples, such as The simulation of TRUNC() in Derby
  • [7]: All Java libraries can be used in Scala, even without any native API.

Conclusion

The above comparison was made at a high level. Without further knowledge of the specific needs for any given project, it doesn’t make sense to go into the details of every library’s capabilities. On a high level, one can draw a clear line between the three products:

jOOQ
jOOQ’s strength is the fact that it supports a huge variety of standard and vendor-specific SQL statements, clauses and functions, and that it is able to simulate these in dialects that do not support them. This makes jOOQ a good choice for applications that want to make extensive use of SQL.

QueryDSL
QueryDSL’s strength is the fact that it supports multiple backends (most importantly JPQL and HQL). It greatly facilitates the construction of criteria queries, reducing JPA’s verbosity.

OneWebSQL™
OneWebSQL™’s strength is probably the fact that you get professional support when you buy a license. I’m looking forward to hearing experiences with this new library

SQL in Scala, where jOOQ could go

I have recently blogged about how simple it is to integrate jOOQ into Scala. See the full blog post here:

https://lukaseder.wordpress.com/2011/12/11/the-ultimate-sql-dsl-jooq-in-scala/

I’m more and more thrilled by that option, as Scala is one of the fastest emerging JVM languages nowadays. The plain integration of a Java library in Scala leaves some open questions. jOOQ knows the “val()” operator or method, to create bind values. See the manual here:

http://www.jooq.org/manual/JOOQ/BindValues/

This operator cannot be used in Scala, as Scala declares “val” as a reserved word. I’ve had similar issues in Java before, when trying to use “case” or “else” in the API, which is not possible either. The path of least resistance is to overload or re-name those methods. “val” was overloaded as “value” in jOOQ 2.0.1. “case” and “else” were re-named a long time ago as “decode” (from Oracle’s DECODE function), and “otherwise” (as in XSL).

So for a full-fledged integration in Scala, jOOQ should be wrapped in a new API called scOOQ ;-). This new API should take Scala’s language features into account in order to make working with jOOQ a lot easier. This could be the chance to re-engineer some of the API and make all API methods uppercase, as is usual with SQL. With Scala’s ability of omitting syntax elements, such as “.” and “()”, the API could then declare one-word methods, such as in this Java example:

MERGE().INTO(MY_TABLE)
       .USING(SOURCE_TABLE)
       .ON(MY_TABLE.ID.equal(SOURCE_TABLE.ID))
       .WHEN().MATCHED().THEN().UPDATE()
         .SET(ID, 1)
         .SET(DATA, "Data")
       .WHEN().NOT().MATCHED().THEN().INSERT(MY_TABLE.ID, MY_TABLE.DATA)
       .VALUES(1, "Data")

While in Java, this looks quite nasty and verbose, in Scala it could be very nice! The below statement should compile in Scala if the API was declared as such:

MERGE INTO MY_TABLE
      USING (SOURCE_TABLE)
      ON (MY_TABLE.ID equal SOURCE_TABLE.ID)
      WHEN MATCHED THEN UPDATE
        SET (ID, 1)
        SET (DATA, "Data")
      WHEN NOT MATCHED THEN INSERT (MY_TABLE.ID, MY_TABLE.DATA)
      VALUES (1, "Data")

Convinced? Contributions very welcome! :-)