49

In java8 it was possible to access fields of class java.lang.reflect.Fields using e.g.

Field.class.getDeclaredFields();

In java12 (starting with java9 ?) this returns only a empty array. This doesn't change even with

--add-opens java.base/java.lang.reflect=ALL-UNNAMED

set.

Any ideas how to achieve this? (Appart from the fact that this might be a bad idea, i want to be able to change a "static final" field in my code during junit testing via reflection. This has been possible with java8 by changing the "modifiers"

Field modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(myfield, myfield.getModifiers() & ~Modifier.FINAL);

)

5
  • 2
    I suggest taking a step back and overthinking your whole setup. Why do you want to mutate a static field in the first place? Commented May 8, 2019 at 11:08
  • 2
    @Lino This is because i need to replace a logger by a "testable" Logger implementation. I am quite sure that there are a other approaches to solve this, but i want to understand what has changed, why and where it might be documented... Commented May 8, 2019 at 11:22
  • 2
    IMO you should not test a logger. But the real question is still valid. Commented May 8, 2019 at 11:28
  • 10
    This seems to be the change: bugs.openjdk.java.net/browse/JDK-8210496 Came in JDK 12 Commented May 8, 2019 at 11:29
  • 3
    You can still modify final fields via sun.misc.Unsafe, but as the name implies, it may or not break everything. Commented May 8, 2019 at 12:50

5 Answers 5

63

Why It No Longer Works

The reason this no longer works in Java 12 is due to JDK-8210522. This CSR says:

Summary

Core reflection has a filtering mechanism to hide security and integrity sensitive fields and methods from Class getXXXField(s) and getXXXMethod(s). The filtering mechanism has been used for several releases to hide security sensitive fields such as System.security and Class.classLoader.

This CSR proposes to extend the filters to hide fields from a number of highly security sensitive classes in java.lang.reflect and java.lang.invoke.

Problem

Many of classes in java.lang.reflect and java.lang.invoke packages have private fields that, if accessed directly, will compromise the runtime or crash the VM. Ideally all non-public/non-protected fields of classes in java.base would be filtered by core reflection and not be readable/writable via the Unsafe API but we are no where near this at this time. In the mean-time the filtering mechanism is used as a band aid.

Solution

Extend the filter to all fields in the following classes:

java.lang.ClassLoader
java.lang.reflect.AccessibleObject
java.lang.reflect.Constructor
java.lang.reflect.Field
java.lang.reflect.Method

and the private fields in java.lang.invoke.MethodHandles.Lookup that are used for the lookup class and access mode.

Specification

There are no specification changes, this is filtering of non-public/non-protected fields that nothing outside of java.base should rely on. None of the classes are serializable.

Basically, they filter out the fields of java.lang.reflect.Field so you can't abuse them—as you're currently trying to do. You should find another way to do what you need; the answer by Eugene appears to provide at least one option.


Proper Fix

The proper way to drop a final modifier is to instrument the running program, and have your agent redefine the class. If you do this when the class is first loaded, it's no different than having modified the class file before the JVM was even started. In other words, it's like the final modifier was never present.


Workaround

Obligatory Warning: The developers of Java obviously don't want you to be able to change a final field into a non-final field without actually changing the class file (e.g., by recompiling the source code, instrumentation, etc.). Use any hack at your own risk; it may have unintended side-effects, work only some times, and/or stop working in a future release.

Use java.lang.invoke

The following uses the java.lang.invoke package. For whatever reason, the same restrictions applied to the Reflection API are not applied to the Invoke API (at least up to and including Java 17; continue reading for more information).

The example modifies the EMPTY_ELEMENTDATA final field of the ArrayList class. This field normally contains an empty array that's shared between all ArrayList instances when initialized with a capacity of 0. The below sets the field to {"Hello", "World!"}, and as you can see by running the program, this results in the list instance containing elements that were never added to it.

Java 12 - 17

I tested this on Java 16.0.2 and Java 17.0.3, both downloaded from https://adoptium.net/.

import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;

public class Main {

  private static final VarHandle MODIFIERS;

  static {
    try {
      var lookup = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
      MODIFIERS = lookup.findVarHandle(Field.class, "modifiers", int.class);
    } catch (IllegalAccessException | NoSuchFieldException ex) {
      throw new RuntimeException(ex);
    }
  }

  public static void main(String[] args) throws Exception {
    var emptyElementDataField = ArrayList.class.getDeclaredField("EMPTY_ELEMENTDATA");
    // make field non-final
    MODIFIERS.set(emptyElementDataField, emptyElementDataField.getModifiers() & ~Modifier.FINAL);
    
    // set field to new value
    emptyElementDataField.setAccessible(true);
    emptyElementDataField.set(null, new Object[] {"Hello", "World!"});

    var list = new ArrayList<>(0);

    // println uses toString(), and ArrayList.toString() indirectly relies on 'size'
    var sizeField = ArrayList.class.getDeclaredField("size");
    sizeField.setAccessible(true);
    sizeField.set(list, 2); // the new "empty element data" has a length of 2

    System.out.println(list);
  }
}

Run the code with:

javac Main.java
java --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED Main

Note: I tried to use the "single source file" feature, but that resulted in a ConcurrentModificationException. As pointed out in the comments, this is likely due to some JIT optimization (e.g., the static final field has been inlined, because the JVM does not expect such a field to be able to change).

Output:

[Hello, World!]

Java 18+

Unfortunately, the above results in the following exception on Java 18.0.1 (downloaded from https://adoptium.net/):

Exception in thread "main" java.lang.UnsupportedOperationException
        at java.base/java.lang.invoke.VarForm.getMemberName(VarForm.java:114)
        at Main.main(Main.java:23)

Where line 23 is:

MODIFIERS.set(emptyElementDataField, emptyElementDataField.getModifiers() & ~Modifier.FINAL);
Sign up to request clarification or add additional context in comments.

17 Comments

This approach may break at any time so best not to rely on it. In general, nobody should expect to be able to change a static final field. The long standing recommendation for mocking and other testing tools is to use an agent and drop the final modifier of fields that you want to mock.
There seems to be general pattern of the invoke package not performing the same checks as reflect, e.g. you can instantiate new enum constants without a problem. By the way, as long as your code belongs to the unnamed module, you can even eliminate the warning programmatically via inserting Module java_base = Field.class.getModule(), unnamed = FieldHelper.class.getModule(); java_base.addOpens("java.lang.reflect", unnamed); java_base.addOpens("java.util", unnamed); at the beginning of FieldHelper’s initializer.
When we consider load-time instrumentation, it’s not different to modifying the class files even before starting the JVM, so the Binary Compatibility constraints apply. Unless the final field also was a compile-time constant, it may work through instrumentation.
I’d phrase it like this: “You can not remove the final modifier through Reflection; the only way to remove it, is by actually changing the class”. It doesn’t matter whether you change the persistent class files after compilation or change them on-the-fly through Instrumentation; in either case, you’re actually changing the class, rather than patching some Reflection artifacts. I dug a bit in the JDK 18 code and the interesting point is, even if changing the modifiers field that way worked, it would have no impact on the ability to modify the field.
I suppose, when you use the single-source-file feature, the ArrayList has been used so much already (when compiling your example on-the-fly), that the static final field access has been optimized by the JIT, so the reflection hack has no effect on this field while the change of the size field is within specification (kind of), so it’s still done. Having a size of two but an array of length zero (the already inlined original reference) is an indicator for a concurrent modification. Consistently, when you use System.out.println(Arrays.toString(list.toArray())); you get [null, null].
|
17

I found a way and it worked on JDK 8, 11, 17.

Method getDeclaredFields0 = Class.class.getDeclaredMethod("getDeclaredFields0", boolean.class);
getDeclaredFields0.setAccessible(true);
Field[] fields = (Field[]) getDeclaredFields0.invoke(Field.class, false);
Field modifiers = null;
for (Field each : fields) {
    if ("modifiers".equals(each.getName())) {
        modifiers = each;
        break;
    }
}
assertNotNull(modifiers);

Don't forget to set the following args when using JDK 11 or higher:

--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED

2 Comments

Nice, this indeed works with JDK17! This allows us to upgrade a project to JDK17, and workaround a bug - until our fix is officially integrated into the JDK.
Stops working with JDK 18. The code of this answer still works in that you get a Field instance for the modifiers field that you can manipulate, but it won’t give you write access to non-modifiable final fields.
13

You can't. This was a change done on purpose.

For example, you could use PowerMock and it's @PrepareForTest - under the hood it uses javassist (bytecode manipulation) if you want to use that for testing purposes. This is exactly what that bug in the comments suggests to do.

In other words, since java-12 - there is no way to access that via vanilla java.

Comments

10

This works in JDK 17.

import java.lang.reflect.Field;
import sun.misc.Unsafe;

/**
 * @author Arnah
 * @since Feb 21, 2021
 **/
public class FieldUtil{

    private static Unsafe unsafe;

    static{
        try{
            final Field unsafeField = Unsafe.class.getDeclaredField("theUnsafe");
            unsafeField.setAccessible(true);
            unsafe = (Unsafe) unsafeField.get(null);
        }catch(Exception ex){
            ex.printStackTrace();
        }
    }

    public static void setFinalStatic(Field field, Object value) throws Exception{
        Object fieldBase = unsafe.staticFieldBase(field);
        long fieldOffset = unsafe.staticFieldOffset(field);

        unsafe.putObject(fieldBase, fieldOffset, value);
    }
}
public class YourClass{
    public static final int MAX_ITEM_ROWS = 35_000;
}

FieldUtil.setFinalStatic(YourClass.class.getDeclaredField("MAX_ITEM_ROWS"), 1);

3 Comments

1) public static final int MAX_ITEM_ROWS = 35_000; is a compile-time constant. At each place where MAX_ITEM_ROWS is read, the constant value 35_000 is already inserted at compile-time. 2) Of course, you won’t notice when your example is trying to set the field’s value to 35_000, the same value it already has. Of course, when “no change” is considered a success, it will look like a success everywhere. 3) if you were able to read the value you’ve written, you’d notice that this code failed badly. You are setting an object reference to an int variable. Only Unsafe makes it possible…
I get this on Java 17.0.5+9-LTS-191 java.lang.IllegalArgumentException at java.base/jdk.internal.misc.Unsafe.staticFieldBase0(Native Method) at java.base/jdk.internal.misc.Unsafe.staticFieldBase(Unsafe.java:1128) at jdk.unsupported/sun.misc.Unsafe.staticFieldBase(Unsafe.java:707) at FieldUtil.setFinalStatic(Example.java:24) Even with --add-opens
I confirm that this works in Java 21.
10

It is possible without java.misc.Unsafe on Java 21 with the following Code and JVM parameters:

JVM-Parameters:

--add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-exports=java.base/java.lang.invoke=ALL-UNNAMED --add-exports=java.base/jdk.internal.access=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED

Java-Code

public class HiddenFieldsRevealed {

    private static final Object greeting = "Hello world";

    public static void main(String... args) throws Throwable {

        Field myStaticFinalField = HiddenFieldsRevealed.class.getDeclaredField("greeting");
        myStaticFinalField.setAccessible(true);

        removeFinalness(myStaticFinalField);

        //Logic taken from java.lang.invoke.MethodHandle.unreflectSetter(Field)
        
        //.unreflectSetter(Field) 
        //=> .unreflectField(Field, false)
        //=> lookup.getDirectFieldNoSecurityManager(memberName.getReferenceKind(), f.getDeclaringClass(), memberName);
        //=> .getDirectFieldCommon(refKind, referenceClass, memberName, false)
        
        Class<?> memberNameClass = Class.forName("java.lang.invoke.MemberName");
        
        Constructor<?> memberNameConstructor = memberNameClass.getDeclaredConstructor(Field.class, boolean.class);
        memberNameConstructor.setAccessible(true);

        Object memberNameInstanceForField = memberNameConstructor.newInstance(myStaticFinalField, true);
        
        Field memberNameFlagsField = memberNameClass.getDeclaredField("flags");
        
        memberNameFlagsField.setAccessible(true);
        
        //Manipulate flags to remove hints to it being final
        memberNameFlagsField.setInt(memberNameInstanceForField, (int)memberNameFlagsField.getInt(memberNameInstanceForField) & ~Modifier.FINAL);
        
        Method getReferenceKindMethod = memberNameClass.getDeclaredMethod("getReferenceKind");
        
        getReferenceKindMethod.setAccessible(true);
        
        byte getReferenceKind = (byte)getReferenceKindMethod.invoke(memberNameInstanceForField);

        MethodHandles.Lookup mh = MethodHandles.privateLookupIn(HiddenFieldsRevealed.class, MethodHandles.lookup());

        Method getDirectFieldCommonMethod = mh.getClass().getDeclaredMethod("getDirectFieldCommon", byte.class, Class.class, memberNameClass, boolean.class);
        
        getDirectFieldCommonMethod.setAccessible(true);
        
        //Invoke last method to obtain the method handle

        MethodHandle o = (MethodHandle) getDirectFieldCommonMethod.invoke(mh, getReferenceKind, myStaticFinalField.getDeclaringClass(), memberNameInstanceForField, false);
        
        o.invoke("ModifiedValue");

        System.out.println(greeting);

    }

    public static void removeFinalness(Field field) throws Throwable {
        Method[] classMethods = Class.class.getDeclaredMethods();

        Method declaredFieldMethod = Arrays.stream(classMethods).filter(x -> Objects.equals(x.getName(), "getDeclaredFields0")).findAny().orElseThrow();

        declaredFieldMethod.setAccessible(true);

        Field[] declaredFieldsOfField = (Field[]) declaredFieldMethod.invoke(Field.class, false);

        Field modifiersField = Arrays.stream(declaredFieldsOfField).filter(x -> Objects.equals(x.getName(), "modifiers")).findAny().orElseThrow();

        modifiersField.setAccessible(true);

        modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL);

    }
}

As of Java 21 this works because of the direct call to getDirectFieldCommon(...) that we can fool by modifying the flags on MemberName instance for a field.

Considering that Unsafe.getStaticFieldOffset(...) is deprecated and this workaround is prone to break, I hope that there will be always a way in the future to modify static final fields at runtime because there are some nasty dependencies that are easier to deal with this way than to have the whole ordeal of writing something that instruments the class or to patch the library on compile with a build system just to remove a final where it should not have been in the first place.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.