How to Patch Your IDE to Fix an Urgent Bug

Clock’s ticking. JDK 11 will remove a bunch of deprecated modules through JEP 320, which includes the Java EE modules, which again includes JAXB, a dependency of many libraries, including jOOQ. Thus far, few people have upgraded to Java 9 or 10, as these aren’t LTS releases. Unlike in the old days, however, people will be forced much earlier to upgrade to Java 11, because Java 8 (the free version) will reach end of life soon after Java 11 is released:

End of Public Updates for Oracle JDK 8
As outlined in the Oracle JDK Support Roadmap below, Oracle will not post further updates of Java SE 8 to its public download sites for commercial use after January 2019

So, we library developers must act and finally modularise our libraries. Which is, quite frankly, a pain. Not because of the module system itself, which works surprisingly well. But because of the toolchain, which is far from being production ready. This mostly includes:

It’s still almost not possible to maintain a modularised project in an IDE (I’ve tried Eclipse and IntelliJ, not Netbeans so far) as there are still tons of bugs. Some of which are showstoppers, halting compilation in the IDE (despite compilation working in Maven). For example:

But rather than just complaining, let’s complain and fix it

Let’s fix our own IDE by patching it

Disclaimer: The following procedure assumes that you have the right to modify your IDE’s source and binaries. To my understanding, this is the case with the EPL licensed Eclipse. It may not be the case for other IDEs.

Disclaimer2: Note, as reddit user fubarbazqux so eloquently put it, there are cleaner ways to apply patches (and contribute them) to the Eclipse community, if you have more time. This article just displays a very easy way to do things without spending too much time to figure out how the Eclipse development processes work, internally. It shows a QUICK FIX recipe

The first bug was already discovered and fixed for Eclipse 4.8, but its RC4 version seems to have tons of other problems, so let’s not upgrade to that yet. Instead, let’s apply the fix that can be seen here to our own distribution:

https://github.com/eclipse/eclipse.jdt.core/commit/e60c4f1f36f7efd5fbc1bbc661872b78c6939230#diff-e517e5944661053f0fcff49d9432b74e

It’s just a single line:

How do we do this?

First off, go to the Eclipse Packages Download page:

http://www.eclipse.org/downloads/eclipse-packages

And download the “Eclipse IDE for Eclipse Committers” distribution:

It will contain all the Eclipse source code, which we’ll need to compile the above class. In the new workspace, create a new empty plugin project:

Specify the correct execution environment (in our case Java 10) and add all the Java Development Tools (JDT) dependencies:

Or just add all the available dependencies, it doesn’t really matter.

You can now open the type that you want to edit:

Now, simply copy the source code from the editor and paste it in a new class inside of your project, which you put in the same package as the original (split packages are still possible in this case, yay)

Inside of your copy, apply the desired patch and build the project. Since you already included all the dependencies, it will be easy to compile your copy of the class, and you don’t have to build the entirety of Eclipse.

Now, go to your Windows Explorer or Mac OS X Finder, or Linux shell or whatever and find the compiled class:

This class can now be copied into the Eclipse plugin. How to find the appropriate Eclipse plugin? Just go to your plugin dependencies and check out the location of the class you’ve opened earlier:

Open that plugin from your Eclipse distribution’s /plugins folder using 7zip or whatever zipping tool you prefer, and overwrite the original class file(s). You may need to close Eclipse first, before you can write to the plugin zip file. And it’s always a good idea to make backup copies of the original plugin(s).

Be careful that if your class has any nested classes, you will need to copy them all, e.g.

MyClass.class
MyClass$1.class // Anonymous class
MyClass$Nested.class // Named, nested class

Restart Eclipse, and your bug should be fixed!

How to fix my own bugs?

You may not always be lucky to find a bug with an existing fix in the bug tracker as in the second case:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=535927

No problemo, we can hack our way around that as well. Launch your normal Eclipse instance (not the “Eclipse IDE for Eclipse Committers” one) with a debug agent running, by adding the following lines to your eclipse.ini file:

-Xdebug 
-Xnoagent 
-Djava.compile=NONE 
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

Launch Eclipse again, then connect to your Eclipse from your other “Eclipse IDE for Eclipse Committers” instance by connecting a debugger:

And start setting breakpoints wherever you need, e.g. here, in my case:

java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.problem.ProblemHandler.handle(ProblemHandler.java:145)
	at org.eclipse.jdt.internal.compiler.problem.ProblemHandler.handle(ProblemHandler.java:226)
	at org.eclipse.jdt.internal.compiler.problem.ProblemReporter.handle(ProblemReporter.java:2513)
	at org.eclipse.jdt.internal.compiler.problem.ProblemReporter.deprecatedType(ProblemReporter.java:1831)
	at org.eclipse.jdt.internal.compiler.problem.ProblemReporter.deprecatedType(ProblemReporter.java:1808)
	at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.checkAndRecordImportBinding(CompilationUnitScope.java:960)
	at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInImports(CompilationUnitScope.java:471)
	at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:501)
	at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:878)
	at org.eclipse.jdt.internal.compiler.ProcessTaskManager.run(ProcessTaskManager.java:141)
	at java.lang.Thread.run(Unknown Source)

And start analysing the problem like your own bugs. The nice thing is, you don’t have to fix the problem, just find it, and possibly comment out some lines of code if you think they’re not really needed. In my case, luckily, the regression was introduced by a new method that is applied to JDK 9+ projects only:

String deprecatedSinceValue(Supplier<AnnotationBinding[]> annotations) {
    // ...
}

The method will check for the new @Deprecated(since="9") attribute on the @Deprecated annotation. Not an essential feature, so let’s just turn it off by adding this line to the source file:

String deprecatedSinceValue(Supplier<AnnotationBinding[]> annotations) {
    if (true) return;
    // ...
}

This will effectively prevent the faulty logic from ever running. Not a fix, but a workaround. For more details about this specific issue, see the report. Of course, never forget to actually report the issue to Eclipse (or whatever your IDE is), so it can be fixed thoroughly for everyone else as well

Compile. Patch. Restart. Done!

Conclusion

Java is a cool platform. It has always been a very dynamic language at runtime, where compiled class files can be replaced by new versions at any moment, and re-loaded by the class loaders. This makes patching code by other vendors very easy, just:

  • Create a project containing the vendors’ code (or if you don’t have the code, the binaries)
  • Apply a fix / workaround to the Java class that is faulty (or if you don’t have the code, decompile the binaries if you are allowed to)
  • Compile your own version
  • Replace the version of the class file from the vendor by yours
  • Restart

This works with all software, including IDEs. In the case of jOOQ, all our customers have the right to modification, and they get the sources as well. We know how useful it is to be able to patch someone else’s code. This article shows it. Now, I can continue modularising jOOQ, and as a side product, improve the tool chain for everybody else as well.

Again, this article displayed a QUICK FIX approach (some call it “hack”). There are more thorough ways to apply patches / fixes, and contribute them back to the vendor.

Another, very interesting option would be to instrument your runtime and apply the fix only to byte code:

And:

https://www.sitepoint.com/fixing-bugs-in-running-java-code-with-dynamic-attach/

A note on IntelliJ and NetBeans

Again, I haven’t tried NetBeans yet (although I’ve heard its Java 9 support has been working very well for quite a while).

While IntelliJ’s Jigsaw support seems more advanced than Eclipse’s (still with a few flaws as well), it currently has a couple of performance issues when compiling projects like jOOQ or jOOλ. In a future blog post, I will show how to “fix” those by using a profiler, like:

  • Java Mission Control (can be used as a profiler, too)
  • YourKit
  • JProfiler

Profilers can be used to very easily track down the main source of a performance problem. I’ve reported a ton to Eclipse already. For instance, this one:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=474686

Where a lot of time is being spent in the processing of Task Tags, like:

  • TODO
  • FIXME
  • XXX

The great thing about profiling this is:

  • You can report a precise bug to the vendor
  • You can find the flawed feature and turn it off as a workaround. Turning off the above task tag feature was a no-brainer. I’m not even using the feature.

So, stay tuned for another blog post, soon.

Java Rocks More Than Ever

On the TIOBE index, Java and C have been sharing the #1 and #2 rank for a long time now, and with the recent GA release of the JDK 8, things are not going to get any worse for our community.

Java simply rocks! And it’s the best platform to build almost any of your applications, out there.

But why does Java rock so much? Is it the JVM? Is it the backwards-compatibility? Is it the easy syntax? Or the millions of free and commercial software available to build your software? All of this and much more.

The Top 10 Reasons why Java Rocks More Than Ever

ZeroTurnaround’s RebelLabs often publish awesome blog posts, which we can only recommend. In this case, we’ve discovered a very well-written series of blog posts explaining why Java is so great in 10 steps, by ZeroTurnaround’s Geert Bevin. The articles include:

Part 1: The Java Compiler

The compiler is one of the things we take for granted in any language, without thinking about its great features. In Java, unlike C++, you can simply compile your code without thinking too much about linking, optimisation and all sorts of other usual compiler features. This is partially due to the JIT (Just In Time compiler), which does further compilation work at runtime.

Read the full article here

Part 2: The Core API

The JDK’s core API consists of a very solid, stable and well-understood set of libraries. While many people complain about the lack of functionality in this area (resorting to Google Guava or Apache Commons), people often forget that the core API is still the one that is underneath all those extensions. Again, from a C++ perspective, this is a truly luxurious situation.

Read the full article here

Part 3: Open Source

In this section, ZeroTurnaround’s Geert Bevin‘s mind-set aligns well with our own at Data Geekery when it comes to the spirit of Open Source – no matter whether this is about free-as-in-freedom, or free-as-in-beer, the point is that so many things about Java are “open”. We’re all in this together.

Read the full article here

Part 4: The Java Memory Model

Again, a very interesting point of view from someone with a solid C++ background. We’re taking many things for granted as Java has had a very good threading and memory model from the beginning, which was corrected only once in the JDK 1.5 in 2004, and which has built a solid grounds for newer API like actor-based ones, Fork/JOIN, etc.

Read the full article here

Part 5: High-Performance JVM

The JVM is the most obvious thing to talk about it has allowed for so many languages to work on so many hardware environments, and it runs so fast, nowadays!

Read the full article here

Part 6: Bytecode

… and the JVM also rocks because of bytecode, of course. Bytecode is a vendor-independent abstraction of machine code, which is very predictable and can be generated, manipulated, and transformed by various technologies. We’ve recently had a guest post by Dr. Ming-Yee Iu who has shown how bytecode transformations can be used to emulate LINQ in Java. Let’s hear it for bytecode!

Read the full article here

Part 7: Intelligent IDEs

15 years ago, developing software worked quite differently. People can write assembler or C programs with vi or Notepad. But when you’re writing a very complex enterprise-scale Java program, you wouldn’t want to miss IDEs, nowadays. We’ve blogged about various reasons why SQLJ has died. The lack of proper IDE support was one of them.

Read the full article here

Part 8: Profiling Tools

Remember when Oracle released Java Mission Control for free developer use with the JDK 7u40? Profiling is something very very awesome. With modern profilers, you can know exactly where your bottleneck is by simply measuring every aspect of your JVM. You don’t have to guess, you can know. How powerful is that?

Read the full article here

Part 9: Backwards Compatibility

While backwards-compatibility has its drawbacks, too, it is still very impressive how long the Java language, the JVM, and the JDK have existed so far without introducing any major backwards-compatibility regressions. The only thing that comes to mind is the introduction of keywords like assert and enum.

Could you imagine introducing the Java 8 Streams API, lambda expressions, default methods, generics, enums, and loads of other features without ever breaking anything? That’s just great!

Read the full article here

Part 10: Maturity With Innovation

In fact, this article is a summary of all the others, saying that Java has been a very well-designed and mature platform from the beginning without ever ceasing to innovate. And it’s true. With Java 8, a great next step has been published that will – again – change the way the enterprise perceives software development for good.

Read the full article here

Java Rocks More Than Ever

It does, and it’s a great great platform with a bright future for all its community participants.

Top 5 Useful Hidden Eclipse Features

Eclipse is a beast. A device whose mystery is only exceeded by its power. Some would call it a continuum transfunctioner. Others would call it a transmogrifier. Yes, it is so huge, it takes years to master. And then, your manager shows up and tells you: We’re using NetBeans now.

Just kidding. No one uses NetBeans, except for Adam Bien. So let’s have a look at 5 very useful, but hidden Eclipse features.

1: My favourite feature: Favourites

Everyone uses libraries with static methods. Since Java 5, we can static import them, so we no longer have to write things like

SomeVeryImportantUtility.split(string1, string2);

But who is going to static-import all those methods manually in every class referencing them? No one. Because you can define your favourite types and members in the preferences:

Preferences > Favorites

Preferences > Favorites

And then, just start typing and look for auto-completion:

Auto-completion

Auto-completion

The auto-completion will also generate the necessary static import. Very useful when using DSLs with many functions, for instance. Obviously, you will want to make a careful decision, which ones are really your favourite libraries and within those libraries, which are your favourite types. As you’re on the jOOQ blog (or a syndication thereof), let me give you a hint. Always favourite org.jooq.impl.DSL.

2: The awesome block selection mode

I’ve recently blogged about it here. This is so awesome, it merits being mentioned one more time.

Block selection

Notice the highlighted button, the sixth from the left. That’s the awesome “Block Selection Mode” (Alt-Shift-A on Windows, or Ctrl+3 and then typing block selection). It lets you write opening quotes on each selected line at the same position. This is so useful when you have to edit large amounts of almost identical lines.

3: The EGit staging view

Aparently, not every EGit user is aware of this view. As a matter of fact, to me, this view is the most important reason why I’m NOT using the shell commands. Check out this beauty:

Staging view

Staging view

OK, not really a beauty. Reminds me of this comic strip by Eric Burke. But we’re in transmogrifier land and the staging view is filled with changes waiting to be added / committed / pushed once you synchronize a repository with its origin. I can now decide on a line-per-line basis, which changes I want to be added to the index (note I haven’t added the main method). This leads to there being staged (added) changes and unstaged (not yet added) changes. As always in EGit, I can now either directly commit, commit+push, amend+commit, amend+commit+push in one go. Try doing that on the command line.

Now after this first commit, I can now again stage/add the main method in a separate commit. I guess, behind the scenes, this might be implemented using stashing or some other sort of local cache. I don’t care, this is beautiful!

I can probably do much more with this beauty, but that would fill an entire book (I’m waiting for such a book, @niborst, if you’re reading this)

If you didn’t understand any of the above Git talk, try this popular guide, or join me with…

4: Type filters

Yes, because Eclipse’s out-of-the-box autocompletion is nonsense. Yes, it is! No one really wants to call hashCode() or wait() or notifyAll() on an API. Ever. Actually, nowadays, hardly anyone even wants to call wait() or notify() even on a lock object, except if they’re writing the JDK’s concurrency libraries. But luckily someone else is doing that for free, and they’re certainly using vi or emacs or a hex editor, so they don’t mind Object methods.

So then, why is my autocompletion popup filled with this nonsense? Let’s create a class…

So many options? Really?

So many options? Really?

Wow. I thought I had only one method? I don’t mind equals(), although the few times I actually type equals() I can type it out. So let’s remove that stuff:

Preferences > Type filters

Preferences > Type filters

Thanks Eclipse for mentioning awt, too. I wish there was an option to remove awt from the JDK and from our collective memory, entirely. But at least, adding awt to the type filters stops you from having to choose between importing java.util.List (99.9%) and java.awt.List (8.3%). The rounding error is due to the amount of times you had previously chosen the wrong list, accidentally, and wondered why you couldn’t assign an ArrayList. Thanks again, awt. Do also note how my chameleon Windows 8 has changed window colours while taking screenshots. Tiles, what’s next? Anyway back to Eclipse, let’s try auto-completing again…

There can be only one

There can be only one

Better? Better!

Now…

5: Formatter tags

After having spent all that time with block selection, carefully formatting our SQL… bam comes the intern and/or styleguide-nazi and auto-formats all your beautiful source files to a huge one-liner. But not for much longer. Protect your code with easy to type formatter tags:

Preferences > Formatting Options > Off/On Tags

Preferences > Formatting Options > Off/On Tags

Remember to use something concise to protect your carefully crafted SQL, as you might have to type these tags once or twice:

Now protected

Now protected

No one touches that code again. Except the intern who forgot to and/or styleguide-nazi who refuses to apply your formatting settings. And the other intern, who uses NetBeans. Just kidding. No one uses NetBeans.

Again, these tags work wonderfully with DSLs, which are hard to auto-format.

More to come

Eclipes is an awesome beast. Every year, new versions are stacked with new features that we Java developers get for free! You can give back to Eclipse. While I think that the foundation (backed by IBM) might not heavily rely on donations, your best option is to report bugs and feature requests, here:

https://bugs.eclipse.org/bugs

… or if you’re brave, to sign the CLA and start contributing.

There is always room for improvement to this awesome developer device, adding more material and candidates for my next top 5 list.

Eclipse’s Awesome Block Selection Mode

This post is about an awesome Eclipse feature, that is completely underestimated and hidden in the menu. Yet, it is so useful in so many situations. The awesome “Block Selection Mode” which can be toggled through Alt-Shift-A on Windows. Here’s an example challenge for the Block Selection Mode:

Is there any way I can write (copy-paste) nicely-formatted SQL queries in Java string literals using Eclipse?

And my answer:

Step 1: Paste

Paste your formatted SQL statement verbatim into your Java file

Step 2: Write opening quotes

Notice the highlighted button, the sixth from the left. That’s the awesome “Block Selection Mode” (Alt-Shift-A on Windows). It lets you write opening quotes on each selected line at the same position

Step 3: Write closing quotes and concatenation

Apply the same technique for the closing quotes and the concatenation sign (`+`)

Step 4: Fix the semi-colon

No comment needed here.

Awesome Eclipse feature, no?