Stop Manually Importing Your ERD Export into jOOQ

ERD (Entity Relationship Diagrams) are a great way of designing and visualising your database model. There is a variety of vendors offering free and commercial ERD tools. Vertabelo by E-Point is a SaaS product where you can design and manage your database schema online. For instance, the jOOQ example database can be modelled as such:

jOOQ Sample Database

jOOQ Sample Database

.

The most important aspect of such an ERD tool, however, is its import / export functionality. Not only can an existing schema be reverse-engineered, but you can also export it in SQL or XML format. This is great news for early adopters of the upcoming jOOQ 3.5, which will finally support importing file-based schema definitions, e.g. using an XML representation of your INFORMATION_SCHEMA.

XSLT to the rescue

In “modern” times where people put JSON everywhere and XML has become evil, people might have forgotten the power of XML and its tool chain. In this case, it will be very trivial to transform the Vertabelo export format into the jOOQ import format:

Export

<Tables>
    <Table Id="t1">
        <Name>LANGUAGE</Name>
        <Description></Description>
        <Columns>
            <Column Id="c1">
                <Name>ID</Name>
                <Type>integer</Type>
                <Nullable>false</Nullable>
                <PK>true</PK>
            </Column>
            <!-- ... -->

A full export file can be seen here.

Import

<information_schema>
    <schemata>
        <schema>
            <schema_name>PUBLIC</schema_name>
        </schema>
    </schemata>
    <tables>
        <table>
            <table_schema>PUBLIC</table_schema>
            <table_name>LANGUAGE</table_name>
        </table>
        <!-- ... -->
    </tables>
    <columns>
        <column>
            <table_schema>PUBLIC</table_schema>
            <table_name>LANGUAGE</table_name>
            <column_name>ID</column_name>
            <data_type>integer</data_type>
            <ordinal_position>1</ordinal_position>
            <is_nullable>false</is_nullable>
        </column>
        <!-- ... -->
    </columns>
    <table_constraints>
        <table_constraint>
            <constraint_schema>PUBLIC</constraint_schema>
            <constraint_name>PK_LANGUAGE</constraint_name>
            <constraint_type>PRIMARY KEY</constraint_type>
            <table_schema>PUBLIC</table_schema>
            <table_name>LANGUAGE</table_name>
        </table_constraint>
        <!-- ... -->

While the Vertabelo export format organised hierarchically, the jOOQ import format is a flat XML representation of the relevant SQL standard INFORMATION SCHEMA tables, as implemented by a variety of databases such as H2, HSQLDB, MySQL, PostgreSQL, SQL Server. The transformation can be done easily with XSLT, looking something like this:

<xsl:template match="/">
    <xsl:key name="schema" 
        match="/DatabaseModel/Tables/Table/Properties/Property[Name = 'Schema']" use="." />

    <information_schema xmlns="http://www.jooq.org/xsd/jooq-meta-3.5.0.xsd">
        <schemata>
            <xsl:apply-templates 
                select="/DatabaseModel/Tables/Table/Properties/Property[Name = 'Schema'][generate-id() = generate-id(key('schema', .)[1])]" 
                mode="schema"/>
        </schemata>
        
        <tables>
            <xsl:apply-templates 
                select="/DatabaseModel/Tables/Table" 
                mode="table"/>
        </tables>
        
        <columns>
            <xsl:apply-templates 
                select="/DatabaseModel/Tables/Table/Columns/Column" 
                mode="column"/>
        </columns>

<!-- ... -->
<xsl:template match="Table" mode="table">
    <table>
        <table_schema>
            <xsl:value-of select="Properties/Property[Name = 'Schema']/Value"/>
        </table_schema>
        <table_name>
            <xsl:value-of select="Name"/>
        </table_name>
    </table>
</xsl:template>

<xsl:template match="Column" mode="column">
    <xsl:variable name="Id" select="@Id"/>
    
    <column>
        <table_schema>
            <xsl:value-of select="ancestor::Table/Properties/Property[Name = 'Schema']/Value"/>
        </table_schema>
        <table_name>
            <xsl:value-of select="ancestor::Table/Name"/>
        </table_name>
        <column_name>
            <xsl:value-of select="Name"/>
        </column_name>
        <data_type>
            <xsl:value-of select="Type"/>
        </data_type>
        <ordinal_position>
            <xsl:value-of select="1 + count(preceding-sibling::Column)"/>
        </ordinal_position>
        <is_nullable>
            <xsl:value-of select="Nullable"/>
        </is_nullable>
    </column>
</xsl:template>

The full XSL file can be seen here.

Configuring Maven

All we need now is to put the Vertabelo export file somewhere in src/main/resources, and transform it with the Codehaus xml-maven-plugin like so:

<!-- The XSLT plugin transforming XML files prior to code generation -->
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>xml-maven-plugin</artifactId>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>transform</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <transformationSets>
            <transformationSet>
                <dir>src/main/resources</dir>
                <includes>
                    <include>vertabelo-export.xml</include>
                </includes>
                <stylesheet>src/main/resources/vertabelo-2-jooq.xsl</stylesheet>
            </transformationSet>
        </transformationSets>
    </configuration>
</plugin>

The output is then available to the jOOQ code generator in the target directory:

<plugin>
    <groupId>org.jooq</groupId>
    <artifactId>jooq-codegen-maven</artifactId>
    <version>${org.jooq.version}</version>

    <executions>
        <execution>
            <id>generate-h2</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <generator>
                    <name>org.jooq.util.DefaultGenerator</name>
                    <database>
                        <!-- We're using the new jOOQ 3.5 XMLDatabase here -->
                        <name>org.jooq.util.xml.XMLDatabase</name>
                        <properties>
                            <property>
                                <key>dialect</key>
                                <value>H2</value>
                            </property>
                            <property>
                                <key>xml-file</key>
                                <value>target/generated-resources/xml/xslt/vertabelo-export.xml</value>
                            </property>
                        </properties>
                        <inputSchema>PUBLIC</inputSchema>
                    </database>
                    <generate>
                        <deprecated>false</deprecated>
                        <instanceFields>true</instanceFields>
                    </generate>
                    <target>
                        <packageName>org.jooq.example.db.h2</packageName>
                        <directory>target/generated-sources/jooq-h2</directory>
                    </target>
                </generator>
            </configuration>
        </execution>
    </executions>
</plugin>

That’s it!

This obviously works with other ERD tools than Vertabelo as well. It should be just as easy to write your own XSL file for your own tool. All you need to do is generate valid XML according to the new http://www.jooq.org/xsd/jooq-meta-3.5.0.xsd schema.

See the pom.xml file here, for details, or download the full example project from GitHub.

jOOQ Newsletter: June 18, 2014 – jOOQ goes JavaOne™

jOOQ Goes JavaOne™

We’re super excited to announce that our CEO and Head of R&D Lukas will be heading to San Francisco this fall to talk about jOOQ at JavaOne™! This is not just great for Data Geekery and jOOQ, but also for the whole Java / SQL ecosystem, as we believe that the Java / SQL integration deserves much more focus at conferences, where buzzwords like Big Data and NoSQL dominate the agenda disproportionally.

From our perspective, the JVM is the best platform for general purpose languages, whereas SQL is the best tool for database interaction – with Oracle SQL being a leader in the industry. So…

2014 will be a great year for Java and SQL

Prior to JavaOne™, we have also been talking at the awesome 33rd Degree and GeekOut conferences, the latter having been hosted by our friends over at ZeroTurnaround who have launched XRebel, a very promising tool to help you find rogue SQL statements in your application. Stay tuned as we’ll be trying out XRebel to compare jOOQ with Hibernate on our blog, soon.

Stay informed about 2014 events on www.jooq.org/news.

Tweet of the Day

Our customers, users, and followers are sharing their love for jOOQ with the world and we can hardly catch up with them! Here are:

Alessio Harri, who is having a great time using OpenJPA (the workhorse) with jOOQ (the artist)

The Vertabelo team over at e-point, who like using jOOQ as a database consumer for their SaaS, collaborative ERD tool.

Petri Kainulainen, who is working around performance problems by getting back in control of his SQL.

Thanks for the shouts, guys! You make the jOOQ experience rock!

Upcoming License Improvements

From our recent negotiations with site license leads, we’ve come to two conclusions that will benefit all of the jOOQ Professional and jOOQ Enterprise customer base.

  • When you buy a car or a TV, you probably don’t run to the supplier every time you encounter a small defect that prevents you from fully enjoying your product. You’ll fix it yourself. We want to do the same in the future. As we trust our customers, and as we already ship our sources, we will soon allow you to implement urgent fixes to jOOQ yourselves, as we believe that this will improve the jOOQ experience for everyone and add further value to your own experience.
  • We understand the requirements of purchasing departments in large organisations. Often, it is easier to purchase a site license from a supplier rather than going through the hassles of adapting workstation-based subscriptions all the time. To respond to this need, we’ll soon publish a discounted, tiered pricing model for large-volume purchases of our perpetual licenses.

Both of these improvements will be deployed to all of our customer base in the beginning of July.

Do you already have any questions regarding what will change / improve? Do not hesitate to contact us.

Community Zone – The jOOQ aficionados have been active!

The jOOQ community has been very active again in the last month. We’re happy to point out these editor’s picks from our radar:

Vlad Mihalcea is a very active blogger on the subject of Hibernate integrations, transaction mangagement and connection pooling performance. We’re looking forward to his future blog posts about how to integrate ORMs with SQL/jOOQ, e.g. by applying emerging architecture patterns such as CQRS. One of his most recent, very interesting blog posts deals precisely with that subject.

Micha Kops has been blogging about a variety of Java tool integrations and has now published this comprehensive and very useful jOOQ tutorial. It is great to see fresh opinions from people just getting to know the platform and blogging about it.

SQL Zone – More common SQL mistakes

Our popular blog series “Top 10 mistakes Java developers make when writing SQL”has been enhanced with yet another must-read article for the Java/SQL community:

Did you know about all those common mistakes? We bet you didn’t. If you haven’t seen enough, see also our previous articles:

SQL Zone – Don’t roll your own OFFSET pagination emulation

One of the great reasons why you should use jOOQ is the fact that jOOQ abstracts away all the hard parts of your SQL dialect. If you’re using Oracle (prior to 12c), SQL Server (prior to 2012), or DB2, you might need to emulate what other databases know as OFFSET pagination. While most people get the simple use-cases right, we’ve tried to outline all of the other issues that may arise when you try to do it yourself in our blog post:

A must-read for all SQL transformation aficionados.

Feedback zone

You’ve read to the end of this newsletter, that’s great! Did you like it? What did we do great? What can we improve? What other subjects would you like us to cover?

We’d love to hear from you, so if you want to reach out to us, just drop a message to contact@datageekery.com. Looking forward to hearing from you!