It is all about the JDBC Basics

We’re very happy to announce a guest post by Marco Behler, who has been blogging about jOOQ in the past.

img31Marco started out in programming (reverse-engineering, actually) and now mainly programmes on the JVM in his day-to-day work. He also always had a sweet tooth for strategy and marketing. Marco Behler GmbH is the result of that hybrid role.

It is all about the JDBC Basics

It is one of the days.

You are reading the Spring documentation’s @Transactional section and still don’t understand the difference between logical and physical transaction scopes. Simultaneously your app throws an
LazyInitializationException and you have no idea why. To top it off you see spontaneous database deadlocks in production and you suspect your connection pool is leaking connections..somehow.

Know what most likely would have helped instead of banging your head against the wall? Spending a couple (literally) of hours on learning the JDBC basics. Let’s find out why:

What are the JDBC basics?

The basics are opening up/closing database connections and then working with transactions. Also understanding how deadlocks, pessimistic and optimistic locking work on a plain JDBC level. A bit of isolation levels and savepoints and then directly on to connection pools and jdbc driver logging. That’s it. Seriously.

Why are the basics so important?

Everything you will encounter in frameworks like Spring, Hibernate, jOOQ etc. builds up on these basics. For example, there are a gazillion topics on the internet regarding Hibernate’s LazyInitializationException and I was scared of that particular exception myself many years ago. But what else would you expect trying to query the database without having a connection to the database open (which is basically all that this exeception is) ?

The same with Spring’s “transaction framework”. There is so much content, or shall we say (F)ear/(U)ncertainty/(D)oubt, out there on how to open up transactions with spring, be it programmatically, with annotations or xml. But what if you knew that under the hood, there is only one way (and actually one line of code) to open up transactions in the JDBC world?

Let me not even get started on the various (mis)configurations of connection pools you see in production in the wild. Or the unawareness of JDBC (driver) logging, which usually leads to debugging in the wild. All basics, which you can master in a couple of hours and which will help you for a lifetime!

Why do people not just learn the basics?

In every middle-sized project there is a ton of technologies involved and there usually is no clear-cut path on how to learn all of them or how they all work together. It simply takes a lot of time and effort to dig through everything.

There’s JPA sessions and JDBC connections and then Spring somehow provides those transactional proxies in 5 different ways and then some other colleague just put jOOQ into the mix, but then somehow my session doesn’t flush and my objects don’t get persisted and the HibernateTransactionManager is not working as expected.

With all of this, I would also hope for my database transactions just to commit – god forbid what happens on rollback :)

But in the end, everything technology mentioned is just a layer on top of JDBC. If you understand transactions or deadlocks or savepoints on the basic level, then Spring or Hibernate or jOOQ will not throw you off.

So what do you recommend ?

If you want to get miles ahead in your day-to-day database programming, you have to start with the basics. Step-by-Step. And then you will see most of your problems automatically evaporate.

Out of my extensive database programming experience, I created an ebook with a ton of ready-to-run exercises, which will take you from Java database novice to expert. At your own pace. You can literally copy the source code of every chapter into your IDE, run it and (hopefully) learn from it. It covers plain JDBC, Spring, Hibernate, jOOQ (soon) and also distributed transactions.

You can read the whole book for free online here, and I would love to get your feedback! I would really like to let the community feedback flow back into future editions of the book. In addition, If you like what you see and the exercises help you, you can also show your support by getting a paid digital version (pdf, epub, mobi).

In any case…

…learn your JDBC basics – and you will profit from them for a lifetime!

jOOQ Newsletter: February 26, 2014

Subscribe to the newsletter here

Tweet of the Day

Our followers, users and customers are shouting their love for jOOQ to the world. Here are:

Ben Hood who is constantly discovering new useful features in jOOQ.

Antoine Comte who is skipping MyBatis to jump directly to jOOQ (in French)

Both guys are absolutely right, of course!

jOOQ 3.4 Outlook

jOOQ 3.3 was an exciting release. But now we’re heading towards the next upgrade, and it’s not going to be any less exciting. Here are the top interesting topics that might be in scope for jOOQ 3.4:

  • IBM Informix support for the jOOQ Professional Edition.
  • Typesafe DDL support for the most common statement types.
  • Transaction management. We’ve been discussing this a lot of times and we would love to provide some jOOQ-style Java-8 ready default behaviour. As always, with plenty of options to override the defaults.
  • TABLE types and TABLE-valued functions in all databases. This is already very useful in jOOQ 3.3’s SQL Server integration.
  • SQL 2 jOOQ improvements and a UI to transform SQL to jOOQ code.

As you can see, plenty of goodies coming up! Missing a feature? Write us!

Community work

We’re thrilled to have spotted yet another great article by Vlad Mihalcea, who is discovering the merits of using jOOQ for interaction with stored procedures and functions. If you’re doing a deep SQL integration with your favourite database of choice, stored procedures are a very useful tool in your tool chain. Unfortunately, neither JDBC nor JPA offer any convenience in that area, in a way as jOOQ does.Read the full article here.

Besides, we’re very happy to have found that our friends at RebelLabs are excited with our blog posts, such that they have featured a guest post of ours on their blog. RebelLabs have a tremendous amount of great content from many writers, so we’re very honoured to present our guest post to you.

SQL Zone – 60% of SQL Developers Fail

One of the most important topics when writing SQL is performance, as the database is the bottleneck in most applications. And surprisingly, 60% of all SQL developers will fail this little test performed by the Use The Index, Luke guys.

This blog post is a must-read for all juniors and seniors alike. You never stop learning about correct SQL indexing.

SQL Zone – JDBC and Booleans

Boolean types have been introduced only late in the SQL standard, namely inSQL:1999. Even today, not all databases natively support BOOLEAN or BIT types. Most importantly, we can still wait for them in Oracle for a while.

What does this mean for Java developers striving for a maximum compatibility through JDBC? Read our findings in this blog post here.

Upcoming Events

February has been a bit quiet as we were preparing for the jOOQ 3.3 release, but soon you’ll get another chance at hearing about jOOQ or SQL in general in any of these upcoming events:

With our new SQL-talk, we’ll no longer just spread some jOOQ love, but also some SQL love in general. We believe that SQL deserves more presence in today’s software engineering talks, and who would be better to talk about SQL than us? Are you interested in hosting this talk at your company? Contact us!

Stay informed about 2014 events on www.jooq.org/news.

Java 8 Friday Goodies: Local Transaction Scope

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem. We have blogged a couple of times about some nice Java 8 goodies, and now we feel it’s time to start a new blog series, the…

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub. tweet this

Java 8 Goodie: Local Transaction Scope

The JavaScript folks often abuse anonymous functions to create local scope. Like any other language feature, this can be abused, but in some contexts, local scoping is really awesome. Java also allows for local scoping, although until Java 8, this has been equally cumbersome:

JavaScript

(function() {
    var local = function() { 
            scoping(); 
        },
        scoping = function() { 
            alert('If you really must');
        };

    local();
})();

Java

new Object() {
    void local() {
        scoping();
    }
    void scoping() {
        System.out.println(
            "Ouch, my fingers. Too much typing");
    }
}.local();

Both examples look really awkward, although the JavaScript folks call this a design pattern. No one would create such local scope in Java, even if the two pieces of code are roughly equivalent.

tweet thisAwkwardness can be a design pattern in JavaScript.

Local scoping in Java 8

But with Java 8, everything changes, and so does local scoping. Let’s have a look at how we can creat a local semantic scope for transactions. For this, we’ll create two types. The Transactional interface:

@FunctionalInterface
interface Transactional {
    void run(DSLContext ctx);
}

For the example, we’re going to be using jOOQ to avoid checked exceptions and verbose statement creation. You can replace it by your SQL API of choice. So, jOOQ provides us with a locally scoped ctx object, which implicitly contains the transaction state. This transaction state is generated using a TransactionRunner:

class TransactionRunner {
    private final boolean silent;
    private final Connection connection;

    TransactionRunner(Connection connection) {
        this(connection, true);
    }

    TransactionRunner(Connection connection,
                      boolean silent) {
        this.connection = connection;
        this.silent = silent;
    }

    void run(Transactional tx) {
        // Initialise some jOOQ objects
        final DefaultConnectionProvider c =
            new DefaultConnectionProvider(connection);
        final Configuration configuration =
            new DefaultConfiguration()
                .set(c).set(SQLDialect.H2);

        try {
            // Run the transaction and pass a jOOQ
            // DSLContext object to it
            tx.run(DSL.using(configuration));

            // If we get here, then commit the
            // transaction
            c.commit();
        }
        catch (RuntimeException e) {

            // Any exception will cause a rollback
            c.rollback();
            System.err.println(e.getMessage());

            // Eat exceptions in silent mode.
            if (!silent)
                throw e;
        }
    }
}

The above is framework code, which we’ll write only once. From now on, we can use the above API trivially in our Java programs. For this, we’ll set up a TransactionRunner like such:

public static void main(String[] args) 
throws Exception {
    Class.forName("org.h2.Driver");
    try (Connection c = DriverManager.getConnection(
            "jdbc:h2:~/test-scope-goodies", 
            "sa", "")) {
        c.setAutoCommit(false);
        TransactionRunner silent = 
            new TransactionRunner(c);

        // Transactional code here ...
    }
}

And now, behold the wonders of Java 8!

// This is a transaction
silent.run(ctx -> {
    ctx.execute("drop table if exists person");
    ctx.execute("create table person(" + 
                "  id integer," +
                "  first_name varchar(50)," +
                "  last_name varchar(50)," +
                "  primary key(id)"+
                ")");
});

// And this is also one transaction
silent.run(ctx -> {
    ctx.execute("insert into person" +
                "  values(1, 'John', 'Smith');");
    ctx.execute("insert into person" +
                "  values(1, 'Steve', 'Adams');");
    // Ouch, fails -------^
    // Transaction rolls back
});

// And this is also one transaction
silent.run(ctx -> {
    ctx.execute("insert into person" + 
                "  values(2, 'Jane', 'Miller');");
    // Works, yay!
});

// And this is also one transaction
silent.run(ctx -> {
    ctx.execute("insert into person" +
                "  values(2, 'Anne', 'Roberts');");
    // Ouch, fails -------^
    // Transaction rolls back
});

What do we get from the above? Let’s check:

silent.run(ctx -> {
    System.out.println(
        ctx.fetch("select * from person"));
});

The above program will yield this output:

SQL [insert into person values(1, 'Steve', 'Adams');];
Unique index or primary key violation: "PRIMARY KEY ON PUBLIC.PERSON(ID)"; SQL statement:
insert into person values(1, 'Steve', 'Adams'); [23505-174]
SQL [insert into person values(2, 'Anne', 'Roberts');];
Unique index or primary key violation: "PRIMARY KEY ON PUBLIC.PERSON(ID)"; SQL statement:
insert into person values(2, 'Anne', 'Roberts'); [23505-174]
+----+----------+---------+
|  ID|FIRST_NAME|LAST_NAME|
+----+----------+---------+
|   2|Jane      |Miller   |
+----+----------+---------+

So, our commits and rollbacks worked as expected!

Nested transactions

We can also create nested calls to our TransactionRunner, e.g. when we’re inside methods calling other methods. For this, would have to adapt our TransactionRunner to count the nesting level, and remove the “silent” functionality. On the other hand, it would be very easy to implement savepoint functionality this way. Each time we nest another transaction, we’ll create a new savepoint.

Conclusion

As always in this series, we didn’t invent anything new. All of these things could be done with vanilla Java 7. But the client code of this TransactionRunner certainly wouldn’t look as lean as our lambdas.

Next week in this blog series, we’re going to look at how Java 8 will allow you to define local caching scope very easily, so stay tuned!

More on Java 8

In the mean time, have a look at Eugen Paraschiv’s awesome Java 8 resources page

MyBatis’ Alternative Transaction Management

On the jOOQ user group, we’re often being asked how to perform transaction management with jOOQ. And we have an easy answer ready: You don’t do that with jOOQ. You choose your favourite transaction management API, be it:

And the above list is far from being exhaustive. Transaction management is something very delicate, and it certainly should not be imposed by a library whose main purpose is not transaction management, because any such library / framework will provide you with at most a very leaky abstraction of its transaction model. In other words, if you just slightly want to deviate from “the standard” model (e.g. as imposed by Hibernate), you will suffer greatly, as soon as you want to run 2-3 queries outside of Hibernate – e.g. batch or reporting statements through jOOQ.

MyBatis’ Alternative Transaction Management

MyBatis is a SQL templating engine that provides a couple of features on top of alternative templating engines, such as Velocity, or StringTemplate. One of these features built on top of templating is precisely transaction management, as can be seen in the docs.

From what we can read in the docs, it looks as though MyBatis’ transaction managers can be overriden by Spring, for instance. However, it is not easy to see how this is done. In fact, given that MyBatis also solves Connection pooling (for which there are also very viable alternatives, such as c3p0 and DBCP), and mapping (which could be solved more easily with custom transformers, such as offered by Spring’s JdbcTemplate, or jOOQ’s RecordMapper).

As many frameworks, MyBatis tries to solve problems outside its core scope, which is SQL templating. While this may be a good thing as you only rely on a single dependency, it is also quite a lock-in, in case you have a more complex model. In the case of transaction management, we believe that this was not a good idea by MyBatis.

Thoughts from MyBatis users?

A nice way of using jOOQ with Spring

This blog post is outdated. For a more up-to-date example of how to integrate jOOQ with Spring, please consider the relevant sections of the jOOQ manual!

A nice way of using jOOQ with Spring was recently discussed on Stack Overflow by Adam Gent:
http://adamgent.com/post/31128631472/getting-jooq-to-work-with-spring-correctly

The essence of it was given here in this gist:

package com.snaphop.jooq;

import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;

import javax.sql.DataSource;

import org.jooq.ExecuteContext;
import org.jooq.impl.DefaultExecuteListener;
import org.springframework.jdbc.datasource.DataSourceUtils;
import org.springframework.jdbc.support.JdbcUtils;
import org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator;
import org.springframework.jdbc.support.SQLExceptionTranslator;
import org.springframework.jdbc.support.SQLStateSQLExceptionTranslator;

/**
 * Example provided by Adam Gent
 */
public class SpringExceptionTranslationExecuteListener
extends DefaultExecuteListener {

    @Override
    public void start(ExecuteContext ctx) {
        DataSource dataSource = ctx.getDataSource();
        Connection c = DataSourceUtils.getConnection(dataSource);
        ctx.setConnection(c);
    }

    @Override
    public void exception(ExecuteContext ctx) {
        SQLException ex = ctx.sqlException();
        Statement stmt = ctx.statement();
        Connection con = ctx.getConnection();
        DataSource dataSource = ctx.getDataSource();
        // This note and code below comes from
        // JdbcTemplate.execute(StatementCallback)
        // Release Connection early, to avoid potential connection pool
        // deadlock in the case when the exception translator hasn't
        // been initialized yet.
        JdbcUtils.closeStatement(stmt);
        stmt = null;
        DataSourceUtils.releaseConnection(con, dataSource);
        con = null;
        ctx.exception(getExceptionTranslator(dataSource)
                        .translate("jOOQ", ctx.sql(), ex));
    }

    /**
     * Return the exception translator for this instance.
     *
Creates a default {@link SQLErrorCodeSQLExceptionTranslator}
     * for the specified DataSource if none set, or a
     * {@link SQLStateSQLExceptionTranslator} in case of no DataSource.
     * @see #getDataSource()
     */
    public synchronized SQLExceptionTranslator
    getExceptionTranslator(DataSource dataSource) {
        // This method probably does not need to be synchronized but in
        // Spring it was because of a mutable field on the JdbcTemplate.
        // Also I have no idea how expensive it is to create a translator
        // as one will get created on every exception.
        final SQLExceptionTranslator exceptionTranslator;
        if (dataSource != null) {
            exceptionTranslator =
                new SQLErrorCodeSQLExceptionTranslator(dataSource);
        }
        else {
            exceptionTranslator = new SQLStateSQLExceptionTranslator();
        }
        return exceptionTranslator;
    }
}

See the relevant Stack Overflow answer for more details:
http://stackoverflow.com/a/12326885/521799