When performing reflective access to default methods in Java, Google seems to fail us. The solutions presented on Stack Overflow, for instance, seem to work only in a certain set of cases, and not on all Java versions.
This article will illustrate different approaches to calling interface default methods through reflection, as may be required by a proxy, for instance.
TL;DR If you’re impatient, all the access methods exposed in this blog are available in this gist, and the problem is also fixed in our library jOOR.
Proxying interfaces with default methods
The useful java.lang.reflect.Proxy
API has been around for a while. We can do cool things like:
import java.lang.reflect.Proxy; public class ProxyDemo { interface Duck { void quack(); } public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { System.out.println("Quack"); return null; } ); duck.quack(); } }
This just yields:
Quack
In this example, we create a proxy instance that implements the Duck
API through an InvocationHandler
, which is essentially just a lambda that gets called for each method call on Duck
.
The interesting bit is when we want to have a default method on Duck
and delegate the call to that default method:
interface Duck { default void quack() { System.out.println("Quack"); } }
We might be inclined to write this:
import java.lang.reflect.Proxy; public class ProxyDemo { interface Duck { default void quack() { System.out.println("Quack"); } } public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { method.invoke(proxy); return null; } ); duck.quack(); } }
But this will just generate a long long stack trace of nested exceptions (this isn’t specific to the method being a default method. You simply cannot do this):
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) at ProxyDemo.main(ProxyDemo.java:20) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at ProxyDemo.lambda$0(ProxyDemo.java:15) ... 2 more Caused by: java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) ... 7 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at ProxyDemo.lambda$0(ProxyDemo.java:15) ... 8 more Caused by: java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) ... 13 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at ProxyDemo.lambda$0(ProxyDemo.java:15) ... 14 more Caused by: java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) ... 19 more ... ... ... goes on forever
Not very helpful.
Using method handles
So, the original Google search turned up results that indicate we need to use the MethodHandles API. Let’s try that, then!
import java.lang.invoke.MethodHandles; import java.lang.reflect.Proxy; public class ProxyDemo { interface Duck { default void quack() { System.out.println("Quack"); } } public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { MethodHandles .lookup() .in(Duck.class) .unreflectSpecial(method, Duck.class) .bindTo(proxy) .invokeWithArguments(); return null; } ); duck.quack(); } }
That seems to work, cool!
Quack
… until it doesn’t.
Calling a default method on a non-private-accessible interface
The interface in the above example was carefully chosen to be “private-accessible” by the caller, i.e. the interface is nested in the caller’s class. What if we had a top-level interface?
import java.lang.invoke.MethodHandles; import java.lang.reflect.Proxy; interface Duck { default void quack() { System.out.println("Quack"); } } public class ProxyDemo { public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { MethodHandles .lookup() .in(Duck.class) .unreflectSpecial(method, Duck.class) .bindTo(proxy) .invokeWithArguments(); return null; } ); duck.quack(); } }
The almost same code snippet no longer works. We get the following IllegalAccessException:
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) at ProxyDemo.main(ProxyDemo.java:26) Caused by: java.lang.IllegalAccessException: no private access for invokespecial: interface Duck, from Duck/package at java.lang.invoke.MemberName.makeAccessException(MemberName.java:850) at java.lang.invoke.MethodHandles$Lookup.checkSpecialCaller(MethodHandles.java:1572) at java.lang.invoke.MethodHandles$Lookup.unreflectSpecial(MethodHandles.java:1231) at ProxyDemo.lambda$0(ProxyDemo.java:19) ... 2 more
Bummer. When googling further, we might find the following solution, which accesses MethodHandles.Lookup
‘s internals through reflection:
import java.lang.invoke.MethodHandles.Lookup; import java.lang.reflect.Constructor; import java.lang.reflect.Proxy; interface Duck { default void quack() { System.out.println("Quack"); } } public class ProxyDemo { public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { Constructor<Lookup> constructor = Lookup.class .getDeclaredConstructor(Class.class); constructor.setAccessible(true); constructor.newInstance(Duck.class) .in(Duck.class) .unreflectSpecial(method, Duck.class) .bindTo(proxy) .invokeWithArguments(); return null; } ); duck.quack(); } }
And yay, we get:
Quack
We get that on JDK 8. What about JDK 9 or 10?
WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by ProxyDemo (file:/C:/Users/lukas/workspace/playground/target/classes/) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class) WARNING: Please consider reporting this to the maintainers of ProxyDemo WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Quack
Oops. That’s what happens by default. If we run the program with the --illegal-access=deny
flag:
java --illegal-access=deny ProxyDemo
Then, we’re getting (and rightfully so):
Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Unable to make java.lang.invoke.MethodHandles$Lookup(java.lang.Class) accessible: module java.base does not "opens java.lang.invoke" to unnamed module @357246de at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:337) at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281) at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:192) at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:185) at ProxyDemo.lambda$0(ProxyDemo.java:18) at $Proxy0.quack(Unknown Source) at ProxyDemo.main(ProxyDemo.java:28)
One of the Jigsaw project’s goals is to precisely not allow such hacks to persist. So, what’s a better solution? This?
import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodType; import java.lang.reflect.Proxy; interface Duck { default void quack() { System.out.println("Quack"); } } public class ProxyDemo { public static void main(String[] a) { Duck duck = (Duck) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { Duck.class }, (proxy, method, args) -> { MethodHandles.lookup() .findSpecial( Duck.class, "quack", MethodType.methodType( void.class, new Class[0]), Duck.class) .bindTo(proxy) .invokeWithArguments(); return null; } ); duck.quack(); } }
Quack
Great, it works in Java 9 and 10, what about Java 8?
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at $Proxy0.quack(Unknown Source) at ProxyDemo.main(ProxyDemo.java:25) Caused by: java.lang.IllegalAccessException: no private access for invokespecial: interface Duck, from ProxyDemo at java.lang.invoke.MemberName.makeAccessException(MemberName.java:850) at java.lang.invoke.MethodHandles$Lookup.checkSpecialCaller(MethodHandles.java:1572) at java.lang.invoke.MethodHandles$Lookup.findSpecial(MethodHandles.java:1002) at ProxyDemo.lambda$0(ProxyDemo.java:18) ... 2 more
You’re kidding, right?
So, there’s a solution (hack) that works on Java 8 but not on 9 or 10, and there’s a solution that works on Java 9 and 10, but not on Java 8.
A more thorough examination
So far, I’ve just been trying to run different things on different JDKs. The following class tries all combinations. It’s also available in this gist here.
Compile it with JDK 9 or 10 (because it also tries using JDK 9+ API: MethodHandles.privateLookupIn()
), but compile it using this command, so you can also run the class on JDK 8:
javac -source 1.8 -target 1.8 CallDefaultMethodThroughReflection.java
import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodHandles.Lookup; import java.lang.invoke.MethodType; import java.lang.reflect.Constructor; import java.lang.reflect.Method; import java.lang.reflect.Proxy; interface PrivateInaccessible { default void quack() { System.out.println(" -> PrivateInaccessible.quack()"); } } public class CallDefaultMethodThroughReflection { interface PrivateAccessible { default void quack() { System.out.println(" -> PrivateAccessible.quack()"); } } public static void main(String[] args) { System.out.println("PrivateAccessible"); System.out.println("-----------------"); System.out.println(); proxy(PrivateAccessible.class).quack(); System.out.println(); System.out.println("PrivateInaccessible"); System.out.println("-------------------"); System.out.println(); proxy(PrivateInaccessible.class).quack(); } private static void quack(Lookup lookup, Class<?> type, Object proxy) { System.out.println("Lookup.in(type).unreflectSpecial(...)"); try { lookup.in(type) .unreflectSpecial(type.getMethod("quack"), type) .bindTo(proxy) .invokeWithArguments(); } catch (Throwable e) { System.out.println(" -> " + e.getClass() + ": " + e.getMessage()); } System.out.println("Lookup.findSpecial(...)"); try { lookup.findSpecial(type, "quack", MethodType.methodType(void.class, new Class[0]), type) .bindTo(proxy) .invokeWithArguments(); } catch (Throwable e) { System.out.println(" -> " + e.getClass() + ": " + e.getMessage()); } } @SuppressWarnings("unchecked") private static <T> T proxy(Class<T> type) { return (T) Proxy.newProxyInstance( Thread.currentThread().getContextClassLoader(), new Class[] { type }, (Object proxy, Method method, Object[] arguments) -> { System.out.println("MethodHandles.lookup()"); quack(MethodHandles.lookup(), type, proxy); try { System.out.println(); System.out.println("Lookup(Class)"); Constructor<Lookup> constructor = Lookup.class.getDeclaredConstructor(Class.class); constructor.setAccessible(true); constructor.newInstance(type); quack(constructor.newInstance(type), type, proxy); } catch (Exception e) { System.out.println(" -> " + e.getClass() + ": " + e.getMessage()); } try { System.out.println(); System.out.println("MethodHandles.privateLookupIn()"); quack(MethodHandles.privateLookupIn(type, MethodHandles.lookup()), type, proxy); } catch (Error e) { System.out.println(" -> " + e.getClass() + ": " + e.getMessage()); } return null; } ); } }
The output of the above program is:
Java 8
$ java -version java version "1.8.0_141" Java(TM) SE Runtime Environment (build 1.8.0_141-b15) Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode) $ java CallDefaultMethodThroughReflection PrivateAccessible ----------------- MethodHandles.lookup() Lookup.in(type).unreflectSpecial(...) -> PrivateAccessible.quack() Lookup.findSpecial(...) -> class java.lang.IllegalAccessException: no private access for invokespecial: interface CallDefaultMethodThroughReflection$PrivateAccessible, from CallDefaultMethodThroughReflection Lookup(Class) Lookup.in(type).unreflectSpecial(...) -> PrivateAccessible.quack() Lookup.findSpecial(...) -> PrivateAccessible.quack() MethodHandles.privateLookupIn() -> class java.lang.NoSuchMethodError: java.lang.invoke.MethodHandles.privateLookupIn(Ljava/lang/Class;Ljava/lang/invoke/MethodHandles$Lookup;)Ljava/lang/invoke/MethodHandles$Lookup; PrivateInaccessible ------------------- MethodHandles.lookup() Lookup.in(type).unreflectSpecial(...) -> class java.lang.IllegalAccessException: no private access for invokespecial: interface PrivateInaccessible, from PrivateInaccessible/package Lookup.findSpecial(...) -> class java.lang.IllegalAccessException: no private access for invokespecial: interface PrivateInaccessible, from CallDefaultMethodThroughReflection Lookup(Class) Lookup.in(type).unreflectSpecial(...) -> PrivateInaccessible.quack() Lookup.findSpecial(...) -> PrivateInaccessible.quack() MethodHandles.privateLookupIn() -> class java.lang.NoSuchMethodError: java.lang.invoke.MethodHandles.privateLookupIn(Ljava/lang/Class;Ljava/lang/invoke/MethodHandles$Lookup;)Ljava/lang/invoke/MethodHandles$Lookup;
Java 9
$ java -version java version "9.0.4" Java(TM) SE Runtime Environment (build 9.0.4+11) Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode) $ java --illegal-access=deny CallDefaultMethodThroughReflection PrivateAccessible ----------------- MethodHandles.lookup() Lookup.in(type).unreflectSpecial(...) -> PrivateAccessible.quack() Lookup.findSpecial(...) -> PrivateAccessible.quack() Lookup(Class) -> class java.lang.reflect.InaccessibleObjectException: Unable to make java.lang.invoke.MethodHandles$Lookup(java.lang.Class) accessible: module java.base does not "opens java.lang.invoke" to unnamed module @30c7da1e MethodHandles.privateLookupIn() Lookup.in(type).unreflectSpecial(...) -> PrivateAccessible.quack() Lookup.findSpecial(...) -> PrivateAccessible.quack() PrivateInaccessible ------------------- MethodHandles.lookup() Lookup.in(type).unreflectSpecial(...) -> class java.lang.IllegalAccessException: no private access for invokespecial: interface PrivateInaccessible, from PrivateInaccessible/package (unnamed module @30c7da1e) Lookup.findSpecial(...) -> PrivateInaccessible.quack() Lookup(Class) -> class java.lang.reflect.InaccessibleObjectException: Unable to make java.lang.invoke.MethodHandles$Lookup(java.lang.Class) accessible: module java.base does not "opens java.lang.invoke" to unnamed module @30c7da1e MethodHandles.privateLookupIn() Lookup.in(type).unreflectSpecial(...) -> PrivateInaccessible.quack() Lookup.findSpecial(...) -> PrivateInaccessible.quack()
Java 10
$ java -version java version "10" 2018-03-20 Java(TM) SE Runtime Environment 18.3 (build 10+46) Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode) $ java --illegal-access=deny CallDefaultMethodThroughReflection ... same result as in Java 9
Conclusion
Getting this right is a bit tricky.
- In Java 8, the best working approach is the hack that opens up the JDK’s internals by accessing a package-private Lookup constructor. This is the only way to consistently call default methods on both private-accessible and private-inaccessible interfaces from any location.
- In Java 9 and 10, the best working approaches are
Lookup.findSpecial()
(didn’t work in Java 8) or the newMethodHandles.privateLookupIn()
(didn’t exist in in Java 8). The latter is required in case the interfaced is located in another module. That module will still need to open the interface’s package to the caller.
It’s fair to say that this is a bit of a mess. The appropriate meme here is:
According to Rafael Winterhalter (author of ByteBuddy), the “real” fix should go into a revised Proxy API:
I’m not sure if that would solve all the problems, but it should definitely be the case that an implementor shouldn’t worry about all of the above.
Also, clearly, this article didn’t do the complete work, e.g. of testing whether the approaches still work if Duck
is imported from another module:
… which will be a topic of another blog post.
Using jOOR
If you’re using jOOR (our reflection library, check it out here), the upcoming version 0.9.8 will include a fix for this:
https://github.com/jOOQ/jOOR/issues/49
The fix simply uses the unsafe reflection approach in Java 8, or the MethodHandles.privateLookupIn()
approach in Java 9+. You can then write:
Reflect.on(new Object()).as(PrivateAccessible.class).quack(); Reflect.on(new Object()).as(PrivateInaccessible.class).quack();