Lesser-Known SQL Features: DEFAULT VALUES

A lesser-known SQL feature is the DEFAULT keyword, which can be used in INSERT and UPDATE statements. Consider the following table, created using standard SQL syntax:

CREATE TABLE timestamps (
  id INTEGER   GENERATED BY DEFAULT 
               AS IDENTITY(START WITH 1),
  t  TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

  CONSTRAINT pk_values PRIMARY KEY (id)
)

Now, in order to generate a new record in this table, you could either explicitly set a timestamp as such:

INSERT INTO timestamps (t) 
  VALUES (CURRENT_TIMESTAMP);

Or, you just use the handy DEFAULT VALUES syntax:

-- Short syntax
INSERT INTO timestamps DEFAULT VALUES;

-- Explicit syntax
INSERT INTO timestamps (t) 
  VALUES (DEFAULT);
INSERT INTO timestamps (id, t) 
  VALUES (DEFAULT, DEFAULT);

The same can be done in an UPDATE statement:

-- Set all timestamps to CURRENT_TIMESTAMP
UPDATE timestamps SET t = DEFAULT;

SQL Compatibility for DEFAULT VALUES

As always with SQL, things aren’t as bright as the SQL-92 standard specifies. According to the standard, all of the above must be supported. In reality, this can be said:

Standards-compliant databases

These databases support the standard, fully

Almost compliant databases

These databases support the DEFAULT keyword, but not the DEFAULT VALUES clause for insert statements:

  • Access
  • DB2
  • Derby (we have created DERBY-6444 for this)
  • MariaDB
  • MySQL
  • Oracle

Support for DEFAULT VALUES in jOOQ 3.3

jOOQ 3.3 will support the DEFAULT VALUES syntax and also a very useful variant of it, when combining inserting DEFAULT values with returning them after the insert:

DSL.using(configuration)
   .insertInto(TIMESTAMPS)
   .defaultValues()
   .returning(TIMESTAMPS.ID, TIMESTAMPS.T)
   .fetch();

The above query will not only insert a new record with default values, but also return those values to your Java program for further processing. As with most jOOQ API elements, the above statement will transparently work with all databases, either through native syntax:

  • DB2: SELECT .. FROM FINAL TABLE (INSERT ..)
  • PostgreSQL: INSERT .. RETURNING

… or through JDBC’s Statement.getGeneratedKeys().

How to simulate MySQL’s INSERT statement extensions

I have previously posted about the SQL MERGE statement, and how powerful it is here:
https://blog.jooq.org/2011/11/29/arcane-magic-with-the-sql2003-merge-statement/

Unfortunately, not all databases support this statement. Also, very often it is quite a bit of overkill, when what you want to do is to simply INSERT or UPDATE a single record, depending on whether it already exists.

MySQL’s nice INSERT statement extensions

MySQL has a couple of very nice extensions to the INSERT statement. Some of them are also supported by the CUBRID database. Let’s have a look at two features of the (simplified) syntax definition:

INSERT [IGNORE]
    [INTO] tbl_name [(col_name,...)]
    VALUES (expr,...),(...),...
    [ ON DUPLICATE KEY UPDATE
      col_name=expr
        [, col_name=expr] ... ]

 
Essentially, we’re looking at two keywords / clauses:

  1. IGNORE: This will cause the INSERT statement to silently fail with a warning, if we’re about to insert a duplicate key
  2. ON DUPLICATE KEY UPDATE: This will cause the INSERT statement to perform an update on all affected records, if we’re about to insert a duplicate key

The full documentation can be seen here:
http://dev.mysql.com/doc/refman/5.5/en/insert.html

How jOOQ models these statements in other dialects

jOOQ’s API conveniently allows for using those MySQL features directly in insert statements. An example

// Insert an author who has recently married and thus
// changed their last name. If we already have that author, then
// update their last name:

create.insertInto(AUTHOR, AUTHOR.ID, AUTHOR.LAST_NAME)
      .values(3, "Smith")
      .onDuplicateKeyUpdate()
      .set(AUTHOR.LAST_NAME, "Smith")
      .execute();

// Alternatively, we may not care if the author already
// exists, we just want to insert them without failure:

create.insertInto(AUTHOR, AUTHOR.ID, AUTHOR.LAST_NAME)
      .values(3, "Smith")
      .onDuplicateKeyIgnore()
      .execute();

Note how jOOQ doesn’t model the IGNORE keyword as a keyword between INSERT [IGNORE] INTO. This keeps the public API a bit more concise, while at the same time being more consistent with pre-existing functionality, both in the INSERT syntax, as well as in the Loader API:

https://blog.jooq.org/2011/08/23/loading-csv-data-with-jooq-2/

Now, databases that do not support these syntax elements can simulate them using MERGE. The above statements will render something like this on an Oracle database:

-- ON DUPLICATE KEY UPDATE statement
merge into "AUTHOR" 
using (select 1 from dual) 
on ("AUTHOR"."ID" = 3) 
when matched then update set "LAST_NAME" = 'Smith' 
when not matched then insert ("ID", "LAST_NAME") values (3, 'Smith')

-- ON DUPLICATE KEY IGNORE statement
merge into "AUTHOR" 
using (select 1 from dual) 
on ("AUTHOR"."ID" = 3) 
when not matched then insert ("ID", "LAST_NAME") values (3, 'Smith')

Conclusion

These practices allow you to write short and concise code using the jOOQ API while still being able to target several backend databases optimally using efficient SQL statements. This will help you increase both your development as well as your runtime performance!

Postgres INSERT .. RETURNING clause and how this can be simulated in other RDBMS

One of jOOQ’s major features is to take the most useful SQL constructs and clauses from any RDBMS and make them available to other SQL dialects, as well. This had been done previously with MySQL’s INSERT .. ON DUPLICATE KEY UPDATE construct, which can be easily simulated with the more powerful SQL standard MERGE statement, or using procedural SQL blocks where this is supported.

A recent request made me think about Postgres’ INSERT .. RETURNING clause, which is probably the most intuitive and concise way of returning generated keys from an insert statement. The importance of doing that becomes clear in the context of a jOOQ UpdatableRecord, which, when inserted, should refresh its IDENTITY, or Primary Key value. With jOOQ 1.6.4, this was done in a bit “risky” way, by just fetching the MAX(PK) value from the table, immediately after insert. Obviously, this can go terribly wrong in a highly concurrent system, when other transactions are committed earlier.

But how to map INSERT .. RETURNING to other RDBMS? Here’s a quick overview over what’s supported in which database / JDBC driver:

Vendor-specific SQL syntax support

On Postgres and DB2, you can also execute this for INSERT statements:

ResultSet rs = statement.executeQuery();

The SQL syntax to fetch a java.sql.ResultSet from an INSERT statement works like this:

-- Postgres
INSERT INTO .. RETURNING *

-- DB2
SELECT * FROM FINAL TABLE (INSERT INTO ..)

Oracle also knows of a similar clause

INSERT INTO .. RETURNING INTO ?, ?, ...

This PL/SQL extension cannot be executed easily with standard JDBC, though. Either, it has to be wrapped in a PL/SQL block and executed as a java.sql.CallableStatement, or by casting the prepared statement into an OraclePreparedStatement and registering return values, as described here:

http://stackoverflow.com/questions/682539/return-rowid-parameter-from-insert-statement-using-jdbc-connection-to-oracle

Optimal JDBC support

Some RDBMS have “optimal” support for returning values after an INSERT statement. By “optimal”, I mean that all table columns can be returned, regardless of whether they are actual keys or not. These RDBMS are: HSQLDB, Oracle, DB2.

If this is supported by the JDBC driver, then the simulation of the Postgres INSERT .. RETURNING clause is very simple, as the requested fields can be passed at prepared statement initialisation time:

// Watch out for case-sensitivity!
String[] columnNames = // [...] transform RETURNING clause
PreparedStatement stmt = connection.prepareStatement(sql, columnNames);

Limited JDBC support

Other RDBMS have “limited” support for returning values. This means that only generated IDENTITY (AUTO_INCREMENT) values will be returned. This applies to: Derby, H2, MySQL, SQL Server.

PreparedStatement stmt = connection.prepareStatement(sql,
  Statement.RETURN_GENERATED_KEYS);

If more than the IDENTITY column value is requested in the simulated INSERT .. RETURNING clause, then an additional SELECT statement has to be issued right after. If transactions are properly handled by client code (i.e. the SELECT will run in the same transaction as the INSERT), then no race conditions can occur and the behaviour of this is correct.

No JDBC support

Unfortunately, there are also JDBC drivers that do not support returning values from INSERT statements. The affected RDBMS are: Sybase, SQLite. In a future version of jOOQ, the INSERT .. RETURNING clause can be simulated in three steps:

  1. INSERT
  2. Fetch Sybase @@identity / SQLite last_insert_rowid()
  3. Fetch other requested columns