Grails is a web framework aimed to boost development productivity. One of the main features is domain centric database schema generation. Applications built with Grails are able to update existing schema just before they start. To do this, Grails is using built-in domain mappers or migrations in more advanced cases. The goal of the UWS-jOOQ Grails-plugin is to integrate jOOQ into the existing Grails lifecycle in order to leverage features of jOOQ without compromising the ones provided by Grails.
This article is part of a series brought to you by the Germany based jOOQ integration partnerUWS Software Service (UWS). UWS is specialised in custom software development, application modernisation and outsourcing with a distinct focus on the Java Enterprise ecosystem.
Why should I use jOOQ with Grails?
In enterprise applications we often face issues with Hibernate performance, lack of support of some statements or just too much hassle caused by the Hibernate model. Hibernate’s Query Language HQL often is not sophisticated enough to cope with some requirements. This forces us to use plain SQL, which is not bad and helps solving specific business problems. However in big projects where a larger group of people is involved and a product continuously evolves, type-safety is very precious but is thrown away when using plain SQL. That’s the moment where the jOOQ framework excels and the UWS-jOOQ Grails-Plugin comes into the game.
How can I Integrate jOOQ into Grails?
We tried to provide a simple integration of jOOQ into Grails using Grails built-in dependency resolution. Just add the following line to the plugins section of your BuildConfig.groovy:
Now all the configuration is ready and it is time to get to one of the main features of jOOQ which is type-safe SQL. With the following command jOOQ will generate Java-classes which you want to use when writing SQL as they will give you compiler-based autocompletion:
Now that everything is in place, let’s say you would like to insert a new record into your database via jOOQ in one of your Controllers. It’s as simple as that:
How does the integration of jOOQ with Grails work under the hood?
In the example above you noticed the JooqService which is dependency-injected by Grails. The JooqService is your entrypoint when it comes to using jOOQ within Grails as it is able to pick your datasource and provide the jOOQ DSL context for you. When you have multiple datasources it also allows you to select a different datasource just by providing the name of it:
DSLContext dsl = jooqService.dataSource_custom
Note that autocompletion won’t tell you about the existence of a dataSource_custom field but JooqService will handle that for you.
In version 0.1 we added JooqService but DSLContext have to generated based on your databases schema. So it is important to execute jooq-init command every time you change your Grails domain model since this command compiles your code and executes all migrations so that latest Java-classes are generated on the latest database-schema. Thanks to this approach it is possible to generate structures even from an in-memory H2 database which will not be available right after the application will be shut down.
Best-practices for using jOOQ with Grails
Integrate legacy databases
You might face the situation where you have to connect to some legacy database using the Grails framework. It’s doable, for sure, but you have to create the right Hibernate mapping first or – with some luck – let Hibernate generate the right one for you. With this plugin you can just let jOOQ generate its Java-classes and you’re ready to communicate with the database using a fully type-safe DSL.
Let a database schema change break your code
It’s one of the most precious gifts when you know that something will break early. With using jOOQ in Grails it will happen during compilation time. When you are executing jooq-init, your application is compiled and the Java-classes are regenerated using the latest database schema. If the generated classes change, you will be notified that your code is not able to compile anymore. You can fix your SQL statements and ensure that your application won’t break during runtime.
Keep generated classes in your version control system
We recommend you to check in jOOQs generated classes into your VCS along with the rest of your applications source code. When you are using jOOQ classes to communicate with the database it’s mandatory for compilation to have those classes already defined. Do you use a different approach? Please let us know!
We’re planning to simplify integration even more and attach jooq-init into regular grails compilation process. Also we’d like to make our plugin harder to misuse (see jooqService section) and add the possibility to use jOOQ not only in services or controllers but also in plain Java classes.
Contribution to UWS-jOOQ Grails-Plugin
This software is distributed under the Apache License, Version 2.0. We want to keep this software free and provide services for people
who integrate jOOQ and Grails. If you’re interested in this project feel free to submit issues or pull requests to the project’s git repository.
The following links provide additional information about the UWS-jOOQ Grails-Plugin :
RxJooq, or reactive jOOQ. How does that sound!? Yes, jOOQ is growing to become a hype among SQL and fluent API aficionados. A recent discussion on reddit already puts jOOQ on the same level with Hibernate with more than 10 mentions in answers to the question “Java: What ORM to use”. Our goal has always been for a Java developer to ask themselves at the beginning of a project:
Is this a jOOQ project, or is this a Hibernate project (or both)?
It is too early to announce anything, but at Data Geekery, we’re very interested and thus putting efforts into collaborating with Red Hat to make the jOOQ / Hibernate integration work more seamlessly, so stay tuned for more goodness in that area.
Groovy and Open Source – What it means for us
You may have heard of Pivotal’s recent announcement about their withdrawing sponsorship from the Groovy and Grails ecosystem. This isn’t exactly a surprise to many people as Pivotal’s main focus has shifted towards their PaaS business quite some time ago. The interesting aspect from our perspective is the fact that a whole ecosystem seems to have relied on the benevolence of a single sponsor. Quite a risk!
We think that Open Source should work differently. Open Source is a fine means of offering freemium and (legally) riskless software to potential customers in order to help customers start engaging with a brand. The ultimate vendor goal with Open Source is always upselling. As our valued jOOQ users and jOOQ newsletter and blog readers, we obviously hope that you will eventually understand all the combined SQL value put into jOOQ, and thus upgrade to a commercial jOOQ subscription.
This wasn’t necessarily the case at Pivotal. There is no obvious path from using Groovy (or Grails) to buying Pivotal’s cloud platform solutions. To make things worse, in order to survive, the Groovy platform now depends on a new, arbitrary sponsor whose incentive to sponsor Groovy might be 100% different from Pivotal’s. For the end user, this will not be the same Groovy any more – so it is hard to believe that Groovy will not suffer heavily from any future transition.
We believe that vendors shouldn’t depend on benevolence. We believe that vendors should have a very clear strategy why they’re creating a product, and do everything necessary to satisfy real customer’s needs. So we want to take the opportunity and thank you for being with us, and for making jOOQ (both the Open Source Edition and the Commercial Editions) what it is: A platform valued by both users of Open Source and commercial databases.
More information about our take on Pivotal and Groovy can be found on our blog:
It’s jOONuary! Profit from our 20% Discount Promotion
Speaking of our customers, there has never been a better time to become one!
Your budget for 2015 has been set in stone? You spent too much money on geeky infrastructure during the Holiday Season? Not a problem for your planned jOOQ integration! If you purchase new jOOQ licenses in jOONuary (January 2015), we will offer you a limited-time 20% discount on all price plans. Act quickly!
We’re a Switzerland-based company, and as such are heavily influenced by recent events on the currency exchange markets. The EUR (which is our sale currency) has plummeted almost 20% compared to the CHF (which is our accounting currency).
This affects all of the Swiss export industry, and many companies are starting to take measures. We will not take any measures thus far and continue with our existing EUR-based price model. For our international customers, nothing will change. For our Swiss customers, this means that in addition to the above jOONuary discount, you will now also benefit from a “Euro discount”! Did we say there has never been a better time to become our customer?
jOOQ 3.6 Outlook
The upcoming jOOQ 3.6 will not be less exciting than the previous versions in the least bit. Here is a quick outline of what we’re going to be doing in the upcoming release:
SAP HANA support. We’ve been talking to database vendors in the past, and we continue to do so, maintaining good relationships with the technical and community people at the vendor side. This time, the collaboration initiative came from the vendor directly, and we’ve heard them.
SAP HANA is an emerging cloud SQL and in-memory SQL platform, with a big Java and Scala based tool chain, which constitutes a perfect match for the jOOQ ecosystem. We’re going to support both HANA’s SQL features as well as HANA’s SQLScript features in the jOOQ 3.6 Enterprise Edition. If you’re an SAP HANA user and interested in details, or in a free preview of jOOQ 3.6.0, please contact sales right away. We’re more than happy to provide you with more info.
Nested records and tables. One of the SQL standard’s most underestimated features is the capability of nesting records and tables. In a true ORDBMS, tables (or MULTISETs) can be nested any level deep. If your SQL database supports these features, it is very easy to materialise a nested object graph directly in the database, instead of relying on the JOIN-based workarounds provided by modern ORMs.
Nesting of records can also be very useful when reusing common data structures, such as audit columns (creation_date, creation_user, modification_date, modification_user). JPA supports the @Embedded annotation for this, and we’ll delve into these features as well.
We believe that true MULTISET support will obsolete our competing products’ most important asset: mapping. Once you can declare all mapping already in SQL, you will no longer miss JPA once you’ve migrated to jOOQ.
A new ConverterProvider SPI. Converters are great for supporting custom data types, but having to register them all the time is tedious. What if jOOQ just supported T <-> U conversion right out of the box, for any combination of T and U? We’ll let you register all your favourite converters and jOOQ figures out the conversion path through the converter graph.
Even better PL/SQL support. PL/SQL types are ubiquitous, but they are not easily accessible via JDBC, and thus via jOOQ. We’re researching a variety of possibilities of working around JDBC’s limitations to allow you to use your favourite PL/SQL types: BOOLEAN, RECORD types, perhaps even table types.
Upcoming jOOQ Events
Have you missed one of our talks and presentations in the recent past? No problem at all, we’re back on the road after a short winter break. Here are all of our upcoming events:
Let’s go back to Groovy: There are probably tens of thousands of Groovy developers out there who have simply downloaded Groovy and then never again interacted with the vendor. It probably wouldn’t even be wrong to say that many developers weren’t aware of Pivotal having been the main sponsor behind Groovy. Sure, Groovy is a strong brand, but it is really “everybody’s brand”, and thus: nobody’s brand. Not being a strong brand, it attracted only techies with language interests (and it is a beautiful language to work with, indeed!)
Pivotal’s move to end support of Groovy is a stark reminder that enterprises who depend on FOSS projects should help support them.
Would it have mattered if “we” had supported Groovy?
Perhaps. A Groovy Foundation (similar to the Apache Foundation, or the Eclipse Foundation) might have made Groovy a bit less dependent on Pivotal, and this can still happen. Possibly, a couple of larger companies who depend on Groovy, or Gradle might chime in and become Silver or Gold or Platinum Sponsors, or something like that. Perhaps, Gradleware will seize the opportunity and “buy” Groovy to become THE Groovy company.
But will it work? Does the same work for Typesafe? Can monetising an Open Source language and platform work in times when even C# is now given away for free?
Red Hat can make money off Linux, because Linux is a very complex ecosystem that just requires a support subscription when you’re running it in production. No bank on this planet will ever run a server farm without the vendor promising 24h support with under 1h reaction time. But is the same true for Scala, Groovy? The “critical” work has long been done by the developers. The binaries are built and shipped onto production where operations takes over. Operations couldn’t care less if that binary is built with Groovy, Scala, Java, Ceylon, Kotlin, Fantom, or any of the other gazillion Java alternatives. All operations will ever care about is the JVM, or Weblogic – and the database, of course. And operations is where the long-term subscription money is, not development.
This doesn’t mean that no one should make money from developers. Companies like JetBrains, ZeroTurnaround, or also ourselves, Data Geekery show that it works, on a much smaller scale. But if a company is “selling” a programming language that doesn’t immediately help them upsell their customers to buy their significant other subscriptions, you should be wary as the vendor motivation to produce the programming language product is very unclear – and in the case of Pivotal, “unclear” is not even close to describe the vendor motivation.
Good examples of holistic platform strategies are these, because operations and the end user can immediately drive the decision chain that justifies the language lock-in for the developers:
OK examples are these, although the upselling potential might not be viable enough to maintain a whole ecosystem. We’ll see how it works:
Kotlin -> IntelliJ
Less good examples are these, because the value proposition chain is really not obvious. There is no justification for the language lock-in:
Groovy -> Cloud Platform ??
Scala -> Reactive Programming ??
Ceylon -> RHEL ??
The Business Model
Jamie Allen’s Tweet shows a lot about what’s wrong with many Open Source vendors. While he claims that end users depend on OSS products from their vendors, the opposite is true. The end user can simply fork the OSS product and lead it to a graceful end of life before replacing it. But the vendor really depends on the goodwill and the benevolence of their FOSS communities. The vendor then tries to leverage goodwill to make a weird-sounding upselling between completely unrelated products. This cannot work.
So join us in our endeavours. Make Open Source a business. A viable business, a business driven by the vendor (and by the market, of course). A business that makes sense. A business that involves dual-licensing and reasonable upselling. A business that uses Open Source mainly as a freemium entry point for the actual business.
You can be romantic about F(L)OSS in your heart, that’s OK. But please don’t depend on it. It would be too bad if you don’t succeed, just because you ran out of money from your “sponsors”, because you didn’t care about the business aspect of your product.
The interesting part in Pivotal’s announcement is this one:
The decision to conclude its sponsorship of Groovy and Grails is part of Pivotal’s larger strategy to concentrate resources on accelerating both commercial and open source projects that support its growing traction in Platform-as-a-Service, Data, and Agile development. Pivotal has determined that the time is right to let further development of Groovy and Grails be led by other interested parties in the open source community who can best serve the goals of those projects.
In other words, Groovy is not a viable business for Pivotal. And it’s hard to disagree here. Groovy has never been created with any commercial interests. Like many Open Source projects, Groovy was created in order to make something “better”, mostly for the sake of it being better. Of course it was useful as it introduced a lot of nice features into the Java ecosystem at a time before all these new JVM languages popped up. And before all these new JVM languages finally had an effect on Java-the-language itself.
On the other hand, the Groovy website’s rather geeky look-and-feel has never made it seem as though virtually anyone had any commercial interests in the language or the platform for that matter. I’m not trying to be harsh here, Groovy is an awesome language, created with love. But maintaining an Open Source ecosystem is hard work. It costs a lot of money and effort. And in the case of Groovy, it is just very hard to disagree with the fact that there is probably little money to be made out of it.
How to make money out of Open Source
When we moved on from a purely Open Source jOOQ to a dual-licensed one, we were criticised a lot by people who realised that they might fall into the dual-licensing category who no longer gets to ride for free. This was of course a disappointing evolution for those people. We see it as one step forward for a product that doesn’t just want to implement l’art pour l’art. We believe that we’re adding value on a small scale to a select set of customers with real SQL problems, and we want to continue to do so. Thus, commercial interests are now the driving force behind our developments, and dual-licensing is the easiest way to achieve that on our own small scale.
Many of those who had criticised us claimed that we should create a support-based Open Source business model instead (like Pivotal!). In other words: Let “them” pay for support – whoever “they” are. But that is not a viable model in the long run.
We create fishing poles. We don’t want to compete with our customers, the fishermen.
In software, the vendor of some product shouldn’t commoditize the main driver for innovation: The product. They should sell the product and create an ecosystem and a market for consultants that will be much better at applying the product to some concrete customer’s business. It is a win-win-win situation for everyone:
Vendors get money from licenses
Consultants get money from their specialist knowledge
End-users get a better, cheaper solution with a lower cost of ownership
Although, the consultant is always the one whose work is commoditized in the long run as demand for the product increases, and more consultants pop up trying to make money from their consulting business.
Joel Spolsky has written an extremely interesting Strategy Letter on the idea of commoditizing a complementary product (support, in this case) to increase the demand for the primary product (license, in this case). In the case of a PaaS company like Pivotal, however, we can only guess that even the commoditization of a whole programming language and ecosystem is no longer sustainable enough to increase demand for their PaaS offerings.
If that is the truth, then other platforms like Spring are at the stake as well! If Groovy is not sustainable, why should Spring be?
Am I Groovy?
Suis-je Groovy? To get back to the original claim:
No, I’m not Groovy. Groovy and every other piece of Open Source software that does not in any direct way produce a commercial value for both the vendor and the consumer is doomed to fail in the long run. Open Source software has created a tremendous amount of value in our industry. Companies like ourselves wouldn’t be possible if we’d still pay millions for an operating system license. The question is not whether software is free as in beer or as in freedom. The question is whether anyone has any viable commercial interests in making a particular software element cheap or even free. If they don’t, well, the joke will be on you as the vendor might just stop doing it. Open Source or not.
Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.
Optional: A new Option in Java
So far, we’ve been pretty thrilled with all the additions to Java 8. All in all, this is a revolution more than anything before. But there are also one or two sore spots. One of them is how Java will never really get rid of
In a previous blog post, we have explained the merits of NULL handling in the Ceylon language, which has found one of the best solutions to tackle this issue – at least on the JVM which is doomed to support the null pointer forever. In Ceylon, nullability is a flag that can be added to every type by appending a question mark to the type name. An example:
The main point behind Optional is to wrap an Object and to provide convenience API to handle nullability in a fluent manner. This goes well with Java 8 lambda expressions, which allow for lazy execution of operations. An example:
Optional<String> stringOrNot = Optional.of("123");
// This String reference will never be null
String alwaysAString =
// This Integer reference will be wrapped again
Optional<Integer> integerOrNot =
// This int reference will never be null
int alwaysAnInt = stringOrNot
.map(s -> Integer.parseInt(s))
There are certain merits to the above in fluent APIs, specifically in the new Java 8 Streams API, which makes extensive use of Optional. For example:
The above piece of code will print any number from the Stream onto the console, but only if such a number exists.
Old API is not retrofitted
For obvious backwards-compatibility reasons, the “old API” is not retrofitted. In other words, unlike Scala, Java 8 doesn’t use Optional all over the JDK. In fact, the only place where Optional is used is in the Streams API. As you can see in the Javadoc, usage is very scarce:
This makes Optional a bit difficult to use. We’ve already blogged about this topic before. Concretely, the absence of an Optional type in the API is no guarantee of non-nullability. This is particularly nasty if you convert Streams into collections and collections into streams.
The Java 8 Optional type is treacherous
The worst implication of Optional on its “infected” API is parametric polymorphism, or simply: generics. When you reason about types, you will quickly understand that:
// This is a reference to a simple type:
// This is a reference to a collection of
// the above simple type:
Generics are often used for what is generally accepted as composition. We have a CollectionofString. With Optional, this compositional semantics is slightly abused (both in Scala and Java) to “wrap” a potentially nullable value. We now have:
// This is a reference to a nullable simple type:
// This is a reference to a collection of
// possibly nullable simple types
So far so good. We can substitute types to get the following:
// This is a reference to a simple type:
// This is a reference to a collection of
// the above simple type:
But now enter wildcards and use-site variance. We can write
// No variance can be applied to simple types:
// Variance can be applied to collections of
// simple types:
Collection<? extends T> source;
Collection<? super T> target;
What do the above types mean in the context of Optional? Intuitively, we would like this to be about things like Optional<? extends Number> or Optional<? super Number>. In the above example we can write:
// Read a T-value from the source
T s = source.iterator().next();
// ... and put it into the target
But this doesn’t work any longer with Optional
Collection<Optional<? extends T>> source;
Collection<Optional<? super T>> target;
// Read a value from the source
Optional<? extends T> s = source.iterator().next();
// ... cannot put it into the target
target.add(s); // Nope
… and there is no other way to reason about use-site variance when we have Optional and subtly more complex API.
If you add generic type erasure to the discussion, things get even worse. We no longer erase the component type of the above Collection, we also erase the type of virtually any reference. From a runtime / reflection perspective, this is almost like using Object all over the place!
Generic type systems are incredibly complex even for simple use-cases. Optional makes things only worse. It is quite hard to blend Optional with traditional collections API or other APIs. Compared to the ease of use of Ceylon’s flow-sensitive typing, or even Groovy’s elvis operator, Optional is like a sledge-hammer in your face.
Be careful when you apply it to your API!
One of the main reasons why Optional is still a very useful addition is the fact that the “object-stream” and the “primitive streams” have a “unified API” by the fact that we also have OptionalInt, OptionalLong, OptionalDouble types.
In other words, if you’re operating on primitive types, you can just switch the stream construction and reuse the rest of your stream API usage source code, in almost the same way. Compare these two chains:
In other words, given the scarce usage of these new types in JDK API, the dubious usefulness of such a type in general (if retrofitted into a very backwards-compatible environment) and the implications generics erasure have on Optional we dare say that
The only reason why this type was really added is to provide a more unified Streams API for both reference and primitive types
That’s tough. And makes us wonder, if we should finally get rid of primitive types altogether.
… Optional isn’t Serializable.
Nope. Not Serializable. Unlike ArrayList, for instance. For the usual reason:
Making something in the JDK serializable makes a dramatic increase in our maintenance costs, because it means that the representation is frozen for all time. This constrains our ability to evolve implementations in the future, and the number of cases where we are unable to easily fix a bug or provide an enhancement, which would otherwise be simple, is enormous. So, while it may look like a simple matter of “implements Serializable” to you, it is more than that. The amount of effort consumed by working around an earlier choice to make something serializable is staggering.
… no, don’t tell me you like Perl. Because you don’t. You never did. It does horrible things. It makes your code look like…
Perl made heavy use of operator overloading and used operators for a variety of things. A similar tendency can be seen in C++ and Scala. See also people comparing the two. So what’s wrong with operator overloading?
People never agreed whether Scala got operator overloading right or wrong:
Operator overloading can be good, but mostly isn’t. In Java, we’re all missing better ways to interact with BigDecimal and similar types:
// How it is:
// How it should be:
bigdecimal1 + bigdecimal2 * bigdecimal3
Of course, operator precedence would take place as expected. Unlike C++ or Scala, ideal operator overloading would simply map common operators to common method names. Nothing more. No one really wants API developers to come up with fancy ##-%>> operators.
While Ceylon, Groovy, and Xtend implemented this in a somewhat predictable and useful way, Kotlin is probably the language that has implemented the best standard operator overloading mechanism into their language. Their documentation states:
a + b
a – b
a * b
a / b
a % b
That looks pretty straightforward. Now check this out:
We’ve recently blogged about Ceylon’s awesome language features. But the above Kotlin features are definitely a killer and would remove any other sorts of desires to introduce operator overloading in Java for good.
Let’s hope future Java versions take inspiration from Kotlin, a language that got operator overloading right.
Groovy is not such a typesafe language. When we miss the .on() clause in the above query, Groovy’s Eclipse IDE integration would indicate that the subsequent call to fetchInto() might not work at run time. But Groovy cannot be sure, just as much as the getValue() calls cannot be guaranteed to work in Groovy’s interpretation of what closures are. This is how Eclipse displays the above code:
What’s also interesting to see is that Groovy cannot infer the SAM (Single Abstract Method) interface type that would best match the fetchInto() call. We have to explicitly tell Groovy to coerce the closure to a jOOQ RecordHandler, and inside that RecordHandler, we cannot access the well-known type of r, which would be:
Record3<String, String, String>
Using jOOQ with Groovy is certainly possible, but also not as powerful as with Scala or with Java 8.
Alternative ways of writing SQL with Groovy
Apart from using a SQL query builder like jOOQ (obviously, as this is the jOOQ blog, or a syndication thereof), you can also use other ways of writing SQL in Groovy. The standard way is to use Groovy’s own SQL support, which is a lot more convenient string-based approach than JDBC directly. In fact, Groovy SQL is how JDBC should have been implemented in the first place: