Oracle will remove the internal sun.misc.Unsafe class in Java 9. While most people are probably rather indifferent regarding this change, some other people – mostly library developers – are not. There had been a couple of recent articles in the blogosphere painting a dark picture of what this change will imply:
- Dripstat‘s Removal of sun.misc.Unsafe in Java 9 – A disaster in the making
- Lluis Martinez’s Removal of sun.misc.Unsafe in Java 9 will break Spring, Hibernate
- Lucy Carey‘s Will Removal of .Unsafe Trigger Javapocalypse?
- Rafael Winterhalter‘s Understanding sun.misc.Unsafe
Maintaining a public API is extremely difficult, especially when the API is as popular as that of the JDK. There is simply (almost) no way to keep people from shooting themselves in the foot. Oracle (and previously Sun) have always declared the sun.* packages as internal and not to be used. Citing from the page called “Why Developers Should Not Write Programs That Call ‘sun’ Packages”:
The sun.* packages are not part of the supported, public interface.
A Java program that directly calls into sun.* packages is not guaranteed to work on all Java-compatible platforms. In fact, such a program is not guaranteed to work even in future versions on the same platform.
This disclaimer is just one out of many similar disclaimers and warnings. Whoever goes ahead and uses Unsafe does so … “unsafely“.
What do we learn from this?
The concrete solution to solving this misery is being discussed and still open. A good idea would be to provide a formal and public replacement before removing Unsafe, in order to allow for migration paths of the offending libraries.
But there’s a more important message to all of this. The message is:
When all you have is a hammer, every problem looks like a thumb
Translated to this situation: The hammer is Unsafe and given that it’s a very poor hammer, but the only option, well, library developers might just not have had much of a choice. They’re not really to blame. In fact, they took a gamble in one of the world’s most stable and backwards compatible software environments (= Java) and they fared extremely well for more than 10 years. Would you have made a different choice in a similar situation? Or, let me ask differently. Was betting on AWT or Swing a much safer choice at the time?
If something can somehow be used by someone, then it will be, no matter how obviously they’re gonna shoot themselves in the foot. The only way to currently write a library / API and really prevent users from accessing internals is to put everything in a single package and make everything package-private. This is what we’ve been doing in jOOQ from the beginning, knowing that jOOQ’s internals are extremely delicate and subject to change all the time.
For more details about this rationale, read also:
However, this solution has a severe drawback for those developing those internals. It’s a hell of a package with almost no structure. That makes development rather difficult.
What would be a better Java, then?
Java has always had an insufficient set of visibilities:
publicprotecteddefault(package-private)private
There should be a fifth visibility that behaves like public but prevents access from “outside” of a module. In a way, that’s between the existing public and default visibilities. Let’s call this the hypothetical module visibility.
In fact, not only should we be able to declare this visibility on a class or member, we should be able to govern module inter-dependencies on a top level, just like the Ceylon language allows us to do:
module org.hibernate "3.0.0.beta" {
import ceylon.collection "1.0.0";
import java.base "7";
shared import java.jdbc "7";
}
This reads very similar to OSGi’s bundle system, where bundles can be imported / exported, although the above module syntax is much much simpler than configuring OSGi.
A sophisticated module system would go even further. Not only would it match OSGi’s features, it would also match those of Maven. With the possibility of declaring dependencies on a Java language module basis, we might no longer need the XML-based Maven descriptors, as those could be generated from a simple module syntax (or Gradle, or ant/ivy).
And with all of this in place, classes like sun.misc.Unsafe could be declared as module-visible for only a few JDK modules – not the whole world. I’m sure the number of people abusing reflection to get a hold of those internals would decrease by 50%.
Conclusion
I do hope that in a future Java, this Ceylon language feature (and also Fantom language feature, btw) will be incorporated into the Java language. A nice overview of Java 9 / Jigsaw’s modular encapsulation can be seen in this blog post:
Until then, if you’re an API designer, do know that all disclaimers won’t work. Your internal APIs will be used and abused by your clients. They’re part of your ordinary public API from day 1 after you publish them. It’s not your user’s fault. That’s how things work.
Aren’t they just considering it for now?
Yes they are for Jigsaw. I should’ve mentioned it. Added a link to http://blog.codefx.org/java/dev/features-project-jigsaw-java-9/#Encapsulation for further reference.
Are you saying that the spec for the Java 9 module system (jigsaw) doesn’t incorporate the Ceylon/Fantom feature of exposing APIs to an entire module internally but not to the outside world (in Ceylon parlance, an unshared package with shared members)? I had taken it for granted that they would. How else would Oracle expose sun.misc.* classes to themselves but not to external modules?
To be honest, in fact I’m actually not up to date with the latest state of Jigsaw. So I’m not saying they’re not doing it. I’m saying I hope what they’re doing matches what other languages already have.
Modules will allow just that but without a cool keyword like module.
In all likelihood there will be a module-info.java (like there are package-info.java now) that will list the exported packages. Without IDE support it will not be as obvious as a new keyword but I think it’s just fine.
Egh… module-info.java. Well, at least I can see the consistency. And imports / exports will probably be done via annotations, I suspect?
“Oracle will remove the internal sun.misc.Unsafe class in Java 9” <<< This is a bit misleading. Technically, Unsafe is not removed. It just resides in a private namespace and the upcoming Jigsaw makes it unavailable from the user defined modules.
Thanks for the clarification. So, in addition to the existing reflection-hack, making
theUnsafeinstance accessible, we’ll now also need an additional reflection-hack, making the module accessible? So, things don’t get that much worse then, do they? :)With the module concept you cannot even get hold of the class itsself! sun.misc.Unsafe is invisible to application classloader. Unless you find some way to extract the Unsafe reference from a public Java class it’s hopeless.
I’m sure there’s a security hole somewhere in the instrumentation APIs :)
Hopeless? While the application class loader may not have access, the class loader of the JDK classes sure will. Something like this should work:
So just a bit more reflection in the current code the looks up Unsafe using reflection anyway.
Wow! I’ve just gotten through reading some of the other threads about this on other web sites and I don’t think I’ve ever seen that much emotion from a bunch of programmers outside of a discussion about Star Trek.
I usually claim to be fairly well-read on these things but I have to admit that I wasn’t even aware of the existence of Unsafe before today. You see, I spend my days doing mundane (mostly Java) development using standard APIs instead of doing sexy stuff with forbidden classes like Unsafe. I was intrigued by all the hand-waving, though, which is why I started reading up on this — and I’m a bit surprised and disturbed by what I’ve seen.
First, I think it’s strange that Oracle wants to remove access to a class that its own developers have found so useful and necessary. Oh, right, I see: their thinking is that it’s not that it’s ALWAYS “unsafe” — it IS safe when it’s used by people who are SMARTER THAN ME. Got it. Thanks a lot.
One thing is for sure: this lays bare any lie that Java is open source in the traditional sense, because this is all coming from Oracle. There’s been no grassroots push to make Unsafe “die in a fire” and for the record, the guy who made that comment isn’t (as I’ve seen him described) an engineer — he’s a “Senior Director of Product Management”, or at least so says his LinkedIn page. In other words, he doesn’t spend his days writing code but rather he spends his days attending meetings where people TALK about writing code.
And regarding his often-quoted mailing list post, note that he first claims that there are “years” to migrate off a dependency from Unsafe but then says (more accurately) that, “this is the year to explain where the API is broken and get it straight”. In the span of a single paragraph we went from having “years” to having THIS year (2015) which is well more than halfway over.
Here’s the thing: all those uses of Unsafe (and other APIs that Jigsaw will make unavailable) didn’t just spring up overnight. They’re the result of TWENTY YEARS of people identifying ways that they could add value “if only” the Java APIs were slightly different. And now Oracle is suggesting that most or all of those workarounds can be incorporated into the official API in five months? Not likely. Instead this smells like a New Coke / Windows 8 moment for Java where upgrading will be shunned by most until Oracle admits that it made a mistake and undoes the damage.
Oh, rest assured, there are many other equally emotional topics. We’ve listed them all here:
https://blog.jooq.org/2014/07/25/top-10-very-very-very-important-topics-to-discuss
That assessment is not entirely fair. After all, this particular class is not part of the Java SE, but part of the Oracle (Sun) JDK implementation. But let’s see if this is really just a New Coke / Windows 8 moment or if Jigsaw is really going to add actual value. We’ll see.
“this particular class is not part of the Java SE, but part of the Oracle (Sun) JDK implementation.”
What is being discussed is a change to the “Open JDK”, which is why the discussions are taking place on the Open JDK mailing lists. Oracle’s JDK implementation is just a packaging of that code, which also happens to be the implementation used by most people who need Java SE. It’s code that Sun / Oracle supposedly was going to let be directed by community consensus and that’s my point: Oracle seems to be moving forward with this change despite what is, by all appearances, a consensus that they shouldn’t.
On the one hand Oracle has said, “The Java community gets to determine the direction of this code”, but on the other hand they’re behaving like a dictator instead of a partner — and not a very benevolent dictator. When it was brought up that making the broken behavior the default would annoy a lot of people, Oracle’s response was, “That’s precisely the point.”
https://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-June/004358.html
And this wasn’t from some low-level flunky but from Mark Reinhold. Apparently he subscribes to the “let them eat cake” school of thought.
Go ahead, fork the said “Open JDK”, create a foundation, invest money and personnel and do it better (let’s see how much the “community” will appreciate such efforts in the long run).
Or, eat the cake.
Oh well
Are you still using Hudson for CI?
Yeah, me neither.
Your substitution doesn’t work for many many reasons, but this is the point in the discussion where I’ll politely withdraw and say: You’re probably right. Cheers! :)
Yep, Java’s package private visibility is kind of pointless encapsulation-wise. I agree that there should be a “module” visibility modifier. C#’ “internal” does just that.
Well, package private is useful but with only very narrow cases. In the past, I’ve thought about emulating modules by (ab-)using (static nested) classes as namespaces and making everything package private. But such files tend to get huge :)
Thanks for pointing out C#’s “internal”. C# has gotten so many things right…