This Beats Everything: Koding in the Cloud

OK, now this beats everything I’ve seen so far. I can now code in the cloud with Koding.com. From a first, very quick glance, I get:

  • A VM with a terminal (looks like a Debian distribution)
  • PHP and all sorts of stuff that is already installed
  • An app store for apps like PostgreSQL or MySQL
  • 1.2 GB of free disk space and 2 GB of RAM

Wow. Coding in the cloud. Sounds awesome. What’s next!? And who are they? See the Koding, Inc. blog for their press release about opening up their services to the public:
http://blog.koding.com/2013/08/koding-is-public

VM with a terminal on Koding
VM with a terminal on Koding

Will Another Play-Style Framework Make its Way to Java?

I’ve just discovered the Ninja Web Framework. This one isn’t “yet another framework”, it’s actually heavily based on the very popular Play Framework ideas. It seems to provide a substitute for the latter, since Zenexity and Typesafe have formed an alliance to further support Play primarily in the Scala ecosystem. Some people may feel that this makes the Play+Java combination a second-class citizen. The Ninja Web Framework is maintained by a Berlin-based company called FinalFrontierLabs, who were great fans of Play when they were using it in Java. You have to love their company profile:

Picture Copyright (c) 2013 by FinalFrontierLabs

As some people seem to have regretted Play’s focus on Scala, this framework is certainly one to keep an eye out for in the near future!

Why do We Need RDBMS?

There was this recent Quora question about why we need RDBMS:

Why not just use text files? What can RDBMS do that a simple text file cannot? Or, why not use several different text files to represent different tables?

Heh. Let’s challenge that through a witty comparison (also given as an answer to the above question)…

Short story (not to be taken too seriously):

Some people just put their keys, wallets, make-up, letters, pencils, more make-up, change, and all the other stuff in a huge purse, spending hours to find stuff when we need to catch the train. Stuff, which they might have actually put in that other purse. Let’s call this purse the text file

I like to structure my stuff. My index says: Wallet in the back pocket, key in the front right pocket, mobile phone in the front left pocket, glasses on my nose. Let’s call this structure the RDBMS.

;-)

Long story:

This Quora question is really interesting in this context: Why did relational database technologies gain traction? What were the historical competing technologies?

Essentially, there had been a single most important driving force at the time, pushing RDBMS way ahead of all alternative storage models: Relational algebra itself, designed mostly by Edgar F. Codd, a brilliant computer scientist of his times.

Not only did popular relational database management systems take care of actually managing data, data structures, physical models, transactions, query models, a powerful query language (implemented as SQLjOOQJPQLLINQ to SQLand various other dialects / APIs), referential intergrity, constraint management etc, etc., they were also based on a very very powerful conceptual model and implementation rules (Codd’s 12 rules). The relational data model can easily model almost all business rules.

So, of course you can write your own data management system. Or you use a proven one that does millions of things for you according to very proven rules conceived by very bright people that got very rich with their systems.

Do You View Database Applications as Military Campaigns?

Military Campaigns?? Haha. Let’s meet Capt. DBA, mastering Sun Tsu’s Art of SQL:
http://de.scribd.com/doc/15490992/The-Art-of-SQL

Citing from the book cover:

Do you view database applications as military campaigns?

Do you see data as row upon row upon row of enemy columns to be winnowed down and slashed away? [sic!]

SQL veteran Stéphane Faroult does.

This is a book about SQL, doing marketing by comparing SQL to war. Well, others might claim that SQL is fun and writing SQL with jOOQ is even more fun. So, after Sun Tsu’s Art of SQL, and after the Vietnam of Computer Science, meet now jOOQ: the Peace Treaty between SQL and Java.

Brian Goetz’s Final State of the Lambda Documentation

This week, Brian Goetz has published the final State of the Lambda documentation, which can be seen here:

These are exciting times in Java, as Java 8’s most impactful project has finally stabilised and can be downloaded here:

Amazing Web Applications with PL/SQL and Formspider

In the good old days, dynamic web applications were created using cgi-bin and C. Yes, C as in pre-C++. Today, this might seem odd or even crazy. But why not. And why not create a website using PL/SQL? Check out Formspider, a web framework that connects AJAX requests directly with PL/SQL stored procedure calls. For instance, to generate charts:

http://theformspider.com/API/api_chart.html

Intrigued? Me too! ;-)

Fast File System Operations with Xtend, Lambdas, and ThreadPools

Recently, I’ve blogged about 10 Subtle Best Practices when Coding Java, and I have mentioned that you should start writing SAMs (Single Abstract Method) now, in order to be prepared for Java 8. But there’s another language gem out there, which comes in handy every once in a while, and that’s Eclipse Xtend. Xtend is a “dialect” of the Java language, compiling into Java source code, which then compiles into byte code.

Here’s a quickie showing how easily recursive file system operations can be done with Xtend, Lambdas, and ThreadPools.

class Transform {

  // This is the thread pool performing
  // all the "hard" work
  static ExecutorService ex;

  def static void main(String[] args) {
    // Initialise the thread pool with
    // something meaningful
    ex = Executors::newFixedThreadPool(4);

    // Pass the root directory to the
    // transform method
    val in = new File(...);

    // Recurse into the file transformation
    transform(in);
  }

  def static transform(File in) {

    // Calculate the target file name
    val out = new File(...);

    // Recurse into directories
    if (in.directory) {

      // Pass a FileFilter in the form of an
      // Xtend lambda expression
      for (file : in.listFiles[path |
             !path.name.endsWith(".class")
          && !path.name.endsWith(".zip")
          && !path.name.endsWith(".jar")
        ]) {
        transform(file);
      }
    }
    else {
      // Pass an Xtend lambda expression to
      // the ExecutorService
      ex.submit[ |
        // Read and write could be implemented
        // in Apache Commons IO
        write(out, transform(read(in)));
      ];
    }
  }

  def static transform(String content) {
    // Do the actual string transformation
  }
}

Granted, with Java 8, we’ll get lambdas as well, and that’s awesome. But Xtend has a couple of other nice features that can be seen above:

  • Passing lambdas to a couple of JDK methods, such as File.listFiles() or ExecutorService.submit()
  • Local variable type inference using valvar, or for
  • Method return type inference using def
  • Ability to omit parentheses when passing a lambda to a method
  • Calling getters and setters by convention, e.g. path.name, instead of path.getName(), or in.directory, instead of in.isDirectory()
  • You could also omit semi-colons, although I don’t personally think that’s a good idea.

Xtend is Java 8 already now, which can be very useful for scripts like the above

jOOQ Newsletter September 17, 2013

Subscribe to this newsletter here.

SQL for calculations

SQL can be used for heavy calculations. This doesn’t mean that it has to, of course. Many Java-oriented software architects are reluctant to allow for business logic entering their database. DBA tend to disagree and promote complex logic in database views or stored procedures.

The pros and cons of both sides can be seen in this popular discussion on reddit. Our take on this discussion is a rather pragmatic one, saying: Neither approach is “better”. But not knowing about awesome, often vendor-specific SQL features is a pity. For example, here are two approaches to calculating a running total with Oracle SQL:

What is your experience with SQL for calculations? Reply to this e-mail and tell us your story.

SQL Performance Explained

SQL Performance ExplainedWhen running calculations in the database using SQL, it is always good to know your way around indexing and performance tuning. As jOOQ is a very SQL-centric environment, we would like to promote a very good book about SQL Performance.

SQL databases have become incredibly fast in parsing and executing even very complex SQL statements. But to many SQL developers, writing performing SQL is still a mystery. SQL Performance Explained by Markus Winandcovers 90% of what any SQL developer should know in very simple terms. A must-read for all SQL beginners and pros! Available in English, German, and French.

PostgreSQL 9.3

PostgreSQL 9.3 has been released! This is great news as a wonderful database has just gotten better. Apart from support for materialised views and updatable views, PostgreSQL now also supports the SQL standard LATERAL JOIN, which will soon be supported in jOOQ as well.

More about what’s new in PostgreSQL 9.3 can be seen here.

Google and MariaDB

After RedHat’s alleged switching from MySQL to MariaDB in RHEL, Google now also announces that they will start to migrate away from Oracle’s popular Open Source database. These migrations will further strengthen MariaDB’s position and communities, as large players in the data market will stop contributing to MySQL and contribute to MariaDB instead.

Read more about this migration here.

Why PostgreSQL is so Awesome

Just recently, I’ve blogged about PostgreSQL 9.3 having been released, which is awesome enough as PostgreSQL finally supports materialised views and updatable views. I have then blogged about PostgreSQL’s syntax being a mystery only exceeded by its power, as it allows for treating INSERT and UPDATE statements as table references, when used with the RETURNING clause. This is quite fancy, even if not very useful in everyday SQL.

But what I’m writing about today is colossal:

PostgreSQL predicates are just ordinary expressions

Let this sink in. To PostgreSQL, predicates are just ordinary expressions evaluating to the boolean type. This is documented in the SELECT syntax reference, here:
http://www.postgresql.org/docs/9.3/static/sql-select.html

Citing:

The optional WHERE clause has the general form

WHERE condition

where condition is any expression that evaluates to a result of type boolean […]

It struck me like lightning when I read this Stack Overflow question. You can put a predicate everywhere! Although I knew this before, I had never thought about how awesome this is! You can use predicates in the SELECT clause:

SELECT a, b, c = d, e IN (SELECT x FROM y)
FROM t

You can use predicates in the GROUP BY clause:

SELECT count(*)
FROM t
GROUP BY c = d, e IN (SELECT x FROM y)

You can use predicates in the ORDER BY clause:

SELECT *
FROM t
ORDER BY c = d, e IN (SELECT x FROM y)

You can aggregate predicates using the EVERY aggregate function.

Don’t believe it? See for yourself in this SQLFiddle!

“Ordinary” SQL

In “ordinary” SQL (i.e. standards-compliant), predicates have to be transformed into ordinary value expressions using the CASE clause. Repeating the above examples:

SELECT clause:

SELECT a, b,
       CASE WHEN c = d THEN true ELSE false END,
       CASE WHEN e IN (SELECT x FROM y)
            THEN true ELSE false END
FROM t

GROUP BY clause:

SELECT count(*)
FROM t
GROUP BY CASE WHEN c = d THEN true ELSE false END,
         CASE WHEN e IN (SELECT x FROM y)
              THEN true ELSE false END

ORDER BY clause:

SELECT *
FROM t
ORDER BY CASE WHEN c = d THEN true ELSE false END,
         CASE WHEN e IN (SELECT x FROM y)
              THEN true ELSE false END

Impact for jOOQ

The answer given in the above Stack Overflow question shows how jOOQ can standardise this behaviour by rendering predicates / conditions directly as column expressions where this is supported, while emulating this behaviour using an equivalent CASE expression, otherwise.

Take predicates in the SELECT clause, for instance:

DSL.using(configuration)
   .select(
       T.A, T.B,
       // Transform a jOOQ Condition into a Field:
       field(T.C.eq(T.D)),
       field(T.E.in(select(Y.X).from(Y)))
   )
   .from(T);

Further thoughts

From jOOQ integration test experience, I can say that Derby, H2, HSQLDB, MariaDB, MySQL, PostgreSQL, and SQLite will work this way. While this is awesome from a syntax perspective, do keep an eye on your execution plans to verify that this isn’t going to produce lots of very expensive nested loops…

PostgreSQL Syntax is a Mystery Only Exceeded by its Power

I just ran across this rather powerful PostgreSQL example statement from the manual. It reads

Increment the sales count of the salesperson who manages the account for Acme Corporation, and record the whole updated row along with current time in a log table:

WITH upd AS (
  UPDATE employees 
  SET sales_count = sales_count + 1 
  WHERE id = (
    SELECT sales_person 
    FROM accounts 
    WHERE name = 'Acme Corporation'
  )
  RETURNING *
)
INSERT INTO employees_log 
SELECT *, current_timestamp 
FROM upd;

This database keeps amazing me. A single statement allows for updating one table, taking the updated record(s) including all generated values as input for an insert into another table. But at the same time, PostgreSQL does not yet implement the SQL:2003 MERGE statement.

Crazy database…