JavaEE or Spring? Neither! We Call Out For a Fresh Competitor!

If you’ve been following some key Java people on Twitter or reading the “news” on Reddit, you could not have missed the hilarious quarrel between some Spring and JavaEE evangelists.

First, Jürgen Höller’s provocative article:

“Happy second birthday, Java EE 7! How is it going in production?”

Then, Reza Rahman’s response:

“The Ghosts of Java EE 7 in Production: Past, Present and Future”

And in between, there had been hundreds of equally amusing tweets.

If you haven’t already, take out your popcorn :)

EDIT: This discussion went on further than this blog post. There are also discussions going on:

Troll marketing

At Data Geekery, we’re certainly not unacquainted with the fun and effective technique that we like to refer to as “troll marketing”, e.g. when we recently ranted against the (ab-)use of annotations by both Java EE and Spring, or when we compared Hibernate with the good old Nokia 3310.

It’s childish, yes. But it is also effective. The only reason why it is effective is because by doing so, you’re stirring up a hornets’ nest. You have a point, after all, and you start triggering a very polarised discussion that has no right/wrong answer, which all participants will fail to accept, and thus fight. While they’re fighting, you can put your brand in each and every message. Bingo.

Spring folks do exactly the same thing. If you’ve heard a talk by Josh Long from Pivotal, you will certainly have noticed that much of the talk is just ranting about things like Internet Explorer. If you’re not the kind of person that takes everything personally, seriously, and/or literally, then such a talk can be quite amusing.

The interesting bit here is the fact that the “standards” evangelists react to Spring’s provocative article way too much. At the bottom of Reza’s article, there is a linked presentation by Bert Ertman, in favour of Java EE (and it wouldn’t be wrong to say that Reza thus endorses this talk’s content and style):

The key point I’m trying to make here is not that two opposing technology vendors try to get a hold of their communities, convincing people that their tech is better (that’s what engineers have been doing long before Tesla vs. Edison).

The key point I’m trying to make here is the fact that one of the vendors claims for themselves to be a “standard”, putting themselves (the JCP = Oracle) on eye level with real standards committees like ISO, DIN, w3c, IETF.

JavaEE is a Standard?

It is, in a way. It is a de facto standard, just like Spring. It is a very loose set of small interest groups that are governed mostly by Oracle (and other vendors) who try to promote and establish their interests and technology over alternative, “proprietary” tech.

JavaEE is a “standard” with a long history. It has always been a rather “weak” standard, which is why Spring was created in the first place. The fact that Reza from Oracle reacts so sensitively to Spring’s deliberate provocation can be interpreted as a sign that the “standard” is still weak and endangered by Spring as an alternative.

An interesting comparison could be made to Siebel vs. Salesforce. Siebel was the de facto “standard” in the CRM industry but Salesforce managed to harass Siebel so much that Siebel had to acknowledge Salesforce as a real competitor in its business long before they even remotely met on eye-level, revenue-wise. This was ultimately leading to Siebel’s demise and acquisition by – how ironic – Oracle, where much of the JCP is hosted.

The same thing has been happening with Java EE and Spring. Spring is now so extremely successful in the market, even if completely (= admittedly) proprietary, that Java EE will have to fight hard to get back on top of Spring. Much harder than merely calling out the old and lame “standards” argument. Java EE has to be better and faster than Spring, not just a “standard”. With all the JCP politics going on, I don’t see that happening, though.

So, will Spring win?

My prediction is: Nope.

This whole story just shows us that standards in our Java ecosystem are weak. There are an incredible amount of players on the market. Niche products like Hibernate and jOOQ can shake up the database access market segment. Niche products like GWT, Vaadin, ZK, Spring MVC, Play Framework, etc. can shake up the HTML UI market segment (no way JSF or JSR 371 can prevent further market fragmentation in this area).

Heck, language agnostic technologies like Akka or Vert.X show that not even the Servlet API is really sound and safe. And with Oracle having introduced Nashorn, the whole JavaScript ecosystem might just as well threaten Java EE. The only thing that can be taken for granted for another 5-10 years, I suspect, is the JVM and the Java SE JDK with its libraries, including JDBC.

Our industry is advancing so quickly right now, it seems to be impossible for a highly distributed, rather political committee like the JCP to keep up the pace and deliver the quality and innovation that we’re seeing elsewhere.

In other words, the Java EE “standard” is nothing more than a 5-10 year-behind body of, admittedly, proven technologies. Proven some time ago. Given that a single player like Pivotal has successfully challenged the “standard” for more than 10 years now, I suspect that new players might soon enter the market with new alternatives.

Who will be the new players?

One of them will be the soon-to-be-renamed Typesafe. Why do you think they’re getting rebranded after all? And what do you think they’re trying to prove with Akka + Play + ConductR? They’re trying to prove that you can do software entirely differently on top of the JVM, that’s what they’re trying to prove.

Besides, they’re using the same low-content, high-emotion troll marketing technique that Pivotal / Spring had been so successful with. Observe their use of the evil term “monolith” everywhere (= Java EE, what else):

monolith

Another one of them might be… Microsoft! Why not? Their recent Open Sourcing moves might help establishing .NET on top of Linux, eventually. And once you’re on top of Linux, you might as well get on top of the JVM. The .NET ecosystem is much more coherent than Java’s and this will exert a lot of innovation pressure on our ecosystem.

microsoft+linux

All of this is happening while the JCP is still recovering from the politics behind the game-changer data interchange APIs JSR-353, JSR-367, and JSR-374

Interesting future

The above predictions might or might not become true. But one thing I’m very certain of. We will have even more competition and thus more choices for our infrastructure software in the near future. The current de facto standards will lose market shares. Tough luck for them, better tech for us.

While the evangelists are fighting their little personal fights, I’m looking forward to a very interesting future on the JVM.

Read about a concurring opinion by our friends at Takipi here

Leaky Abstractions, or How to Bind Oracle DATE Correctly with Hibernate

We’ve recently published an article about how to bind the Oracle DATE type correctly in SQL / JDBC, and jOOQ. This article got a bit of traction on reddit with an interesting remark by Vlad Mihalcea, who is frequently blogging about Hibernate, JPA, transaction management and connection pooling on his blog. Vlad pointed out that this problem can also be solved with Hibernate, and we’re going to look into this, shortly.

What is the problem with Oracle DATE?

The problem that was presented in the previous article is dealing with the fact that if a query uses filters on Oracle DATE columns:

// execute_at is of type DATE and there's an index
PreparedStatement stmt = connection.prepareStatement(
    "SELECT * " + 
    "FROM rentals " +
    "WHERE rental_date > ? AND rental_date < ?");

… and we’re using java.sql.Timestamp for our bind values:

stmt.setTimestamp(1, start);
stmt.setTimestamp(2, end);

… then the execution plan will turn very bad with a FULL TABLE SCAN or perhaps an INDEX FULL SCAN, even if we should have gotten a regular INDEX RANGE SCAN.

-------------------------------------
| Id  | Operation          | Name   |
-------------------------------------
|   0 | SELECT STATEMENT   |        |
|*  1 |  FILTER            |        |
|*  2 |   TABLE ACCESS FULL| RENTAL |
-------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(:1<=:2)
   2 - filter((INTERNAL_FUNCTION("RENTAL_DATE")>=:1 AND 
              INTERNAL_FUNCTION("RENTAL_DATE")<=:2))

This is because the database column is widened from Oracle DATE to Oracle TIMESTAMP via this INTERNAL_FUNCTION(), rather than truncating the java.sql.Timestamp value to Oracle DATE.

More details about the problem itself can be seen in the previous article

Preventing this INTERNAL_FUNCTION() with Hibernate

You can fix this with Hibernate’s proprietary API, using a org.hibernate.usertype.UserType.

Assuming that we have the following entity:

@Entity
public class Rental {

    @Id
    @Column(name = "rental_id")
    public Long rentalId;

    @Column(name = "rental_date")
    public Timestamp rentalDate;
}

And now, let’s run this query here (I’m using Hibernate API, not JPA, for the example):

List<Rental> rentals =
session.createQuery("from Rental r where r.rentalDate between :from and :to")
       .setParameter("from", Timestamp.valueOf("2000-01-01 00:00:00.0"))
       .setParameter("to", Timestamp.valueOf("2000-10-01 00:00:00.0"))
       .list();

The execution plan that we’re now getting is again inefficient:

-------------------------------------
| Id  | Operation          | Name   |
-------------------------------------
|   0 | SELECT STATEMENT   |        |
|*  1 |  FILTER            |        |
|*  2 |   TABLE ACCESS FULL| RENTAL |
-------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(:1<=:2)
   2 - filter((INTERNAL_FUNCTION("RENTAL0_"."RENTAL_DATE")>=:1 AND 
              INTERNAL_FUNCTION("RENTAL0_"."RENTAL_DATE")<=:2))

The solution is to add this @Type annotation to all relevant columns…

@Entity
@TypeDefs(
    value = @TypeDef(
        name = "oracle_date", 
        typeClass = OracleDate.class
    )
)
public class Rental {

    @Id
    @Column(name = "rental_id")
    public Long rentalId;

    @Column(name = "rental_date")
    @Type(type = "oracle_date")
    public Timestamp rentalDate;
}

and register the following, simplified UserType:

import java.io.Serializable;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.sql.Types;
import java.util.Objects;

import oracle.sql.DATE;

import org.hibernate.engine.spi.SessionImplementor;
import org.hibernate.usertype.UserType;

public class OracleDate implements UserType {

    @Override
    public int[] sqlTypes() {
        return new int[] { Types.TIMESTAMP };
    }

    @Override
    public Class<?> returnedClass() {
        return Timestamp.class;
    }

    @Override
    public Object nullSafeGet(
        ResultSet rs, 
        String[] names, 
        SessionImplementor session, 
        Object owner
    )
    throws SQLException {
        return rs.getTimestamp(names[0]);
    }

    @Override
    public void nullSafeSet(
        PreparedStatement st, 
        Object value, 
        int index, 
        SessionImplementor session
    )
    throws SQLException {
        // The magic is here: oracle.sql.DATE!
        st.setObject(index, new DATE(value));
    }

    // The other method implementations are omitted
}

This will work because using the vendor-specific oracle.sql.DATE type will have the same effect on your execution plan as explicitly casting the bind variable in your SQL statement, as shown in the previous article: CAST(? AS DATE). The execution plan is now the desired one:

------------------------------------------------------
| Id  | Operation                    | Name          |
------------------------------------------------------
|   0 | SELECT STATEMENT             |               |
|*  1 |  FILTER                      |               |
|   2 |   TABLE ACCESS BY INDEX ROWID| RENTAL        |
|*  3 |    INDEX RANGE SCAN          | IDX_RENTAL_UQ |
------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(:1<=:2)
   3 - access("RENTAL0_"."RENTAL_DATE">=:1 
          AND "RENTAL0_"."RENTAL_DATE"<=:2)

If you want to reproduce this issue, just query any Oracle DATE column with a java.sql.Timestamp bind value through JPA / Hibernate, and get the execution plan as indicated here.

Don’t forget to flush shared pools and buffer caches to enforce the calculation of new plans between executions, because the generated SQL is the same each time.

Can I do it with JPA 2.1?

At first sight, it looks like the new converter feature in JPA 2.1 (which works just like jOOQ’s converter feature) should be able to do the trick. We should be able to write:

import java.sql.Timestamp;

import javax.persistence.AttributeConverter;
import javax.persistence.Converter;

import oracle.sql.DATE;

@Converter
public class OracleDateConverter 
implements AttributeConverter<Timestamp, DATE>{

    @Override
    public DATE convertToDatabaseColumn(Timestamp attribute) {
        return attribute == null ? null : new DATE(attribute);
    }

    @Override
    public Timestamp convertToEntityAttribute(DATE dbData) {
        return dbData == null ? null : dbData.timestampValue();
    }
}

This converter can then be used with our entity:

import java.sql.Timestamp;

import javax.persistence.Column;
import javax.persistence.Convert;
import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
public class Rental {

    @Id
    @Column(name = "rental_id")
    public Long rentalId;

    @Column(name = "rental_date")
    @Convert(converter = OracleDateConverter.class)
    public Timestamp rentalDate;
}

But unfortunately, this doesn’t work out of the box as Hibernate 4.3.7 will think that you’re about to bind a variable of type VARBINARY:

// From org.hibernate.type.descriptor.sql.SqlTypeDescriptorRegistry

    public <X> ValueBinder<X> getBinder(JavaTypeDescriptor<X> javaTypeDescriptor) {
        if ( Serializable.class.isAssignableFrom( javaTypeDescriptor.getJavaTypeClass() ) ) {
            return VarbinaryTypeDescriptor.INSTANCE.getBinder( javaTypeDescriptor );
        }

        return new BasicBinder<X>( javaTypeDescriptor, this ) {
            @Override
            protected void doBind(PreparedStatement st, X value, int index, WrapperOptions options)
                    throws SQLException {
                st.setObject( index, value, jdbcTypeCode );
            }
        };
    }

Of course, we can probably somehow tweak this SqlTypeDescriptorRegistry to create our own “binder”, but then we’re back to Hibernate-specific API. This particular implementation is probably a “bug” at the Hibernate side, which has been registered here, for the record:

https://hibernate.atlassian.net/browse/HHH-9553

Conclusion

Abstractions are leaky on all levels, even if they are deemed a “standard” by the JCP. Standards are often a means of justifying an industry de-facto standard in hindsight (with some politics involved, of course). Let’s not forget that Hibernate didn’t start as a standard and massively revolutionised the way the standard-ish J2EE folks tended to think about persistence, 14 years ago.

In this case we have:

  • Oracle SQL, the actual implementation
  • The SQL standard, which specifies DATE quite differently from Oracle
  • ojdbc, which extends JDBC to allow for accessing Oracle features
  • JDBC, which follows the SQL standard with respect to temporal types
  • Hibernate, which offers proprietary API in order to access Oracle SQL and ojdbc features when binding variables
  • JPA, which again follows the SQL standard and JDBC with respect to temporal types
  • Your entity model

As you can see, the actual implementation (Oracle SQL) leaked up right into your own entity model, either via Hibernate’s UserType, or via JPA’s Converter. From then on, it will hopefully be shielded off from your application (until it won’t), allowing you to forget about this nasty little Oracle SQL detail.

Any way you turn it, if you want to solve real customer problems (i.e. the significant performance issue at hand), then you will need to resort to vendor-specific API from Oracle SQL, ojdbc, and Hibernate – instead of pretending that the SQL, JDBC, and JPA standards are the bottom line.

But that’s probably alright. For most projects, the resulting implementation-lockin is totally acceptable.

SQL incompatibilities: NOT IN and NULL values

This is something where many hours of debugging have been spent in the lives of many SQL developers. The various situations where you can have NULL values in NOT IN predicates and anti-joins. Here’s a typical situation:

with data as (
  select 1 as id from dual union all
  select 2 as id from dual union all
  select 3 as id from dual
)
select * from data
where id not in (1, null)

What do you think this will return? Well, since “dual” indicates an Oracle database, you might say: “an empty result set”. And you would be right for Oracle. In fact, you would be right for any of these databases:

  • DB2
  • Derby
  • H2
  • Ingres
  • Oracle
  • Postgres
  • SQL Server
  • SQLite
  • Sybase

BUT! You would be wrong for any of these ones:

  • HSQLDB
  • MySQL
  • Sybase ASE

Why the discrepancy?

Intuitively, you’d say that all the big ones treat NULL specially in NOT IN predicates, and it is easy to understand, why:

-- This predicate here...
id not in (1, null)

-- Could be seen as equivalent to this one:
id != 1 and id != null

There’s no id that fulfills the above predicate id != null (not even null itself), hence an empty result set. MySQL is known for some strong abuse of SQL standards compliance, so it’s not surprising that they tweaked this syntax as well.

But wait!

HSQLDB 2.0 is one of the most standards-compliant databases out there, could they really have gotten it wrong? Let’s consider the standard: SQL 1992, chapter 8.4 <in predicate>:

<in predicate> ::=
   <row value constructor>
      [ NOT ] IN <in predicate value>

<in predicate value> ::=
   <table subquery>
      | <left paren> <in value list> <right paren>

<in value list> ::=
   <value expression> { <comma> <value expression> }...

 

And then, further down:

2) Let RVC be the <row value constructor> and 
   let IPV be the <in predicate value>.

3) The expression
     RVC NOT IN IPV

   is equivalent to
     NOT ( RVC IN IPV )

4) The expression
     RVC IN IPV

   is equivalent to
     RVC = ANY IPV

 

So in fact, this can be said:

ID NOT IN (1, NULL) is equivalent to
NOT (ID IN (1, NULL)), equivalent to
NOT (ID = ANY(1, NULL)), equivalent to
NOT (ID = 1 OR ID = NULL), equivalent to
NOT (ID = 1) AND NOT (ID = NULL), which is always UNKNOWN

Conclusion

It looks for once, that HSQLDB 2.0 is not standards-compliant in that evaluating the expression inside NOT() before applying NOT() has a different outcome from transforming NOT() into a normalised boolean expression, and then evaluating the expression. For SQL developers, all of this can just mean:

Keep NULL out of NOT IN predicates or be doomed!