JEP 544: Ahead-of-Time Code Compilation

Summary Improve startup and warmup time by making optimized native code for an application instantly available when the HotSpot Java Virtual Machine starts. Achieve this by compiling application code to native code in a training run, storing the native code in the AOT cache for use in subsequent production runs. If the workload changes in production, regenerate native code dynamically for continued peak performance, providing the best of both ahead-of-time (AOT) and just-in-time (JIT) compilation. Goals Enable applications to achieve peak performance more quickly. Enable applications to sustain peak performance even as workloads change. Do not require any change to the code of applications, libraries, or frameworks. Do not require any change to the configuration of HotSpot, beyond requesting the use of the AOT cache. Continue to support the Serial, Parallel, G1, and ZGC garbage collectors. Do not introduce new AOT workflows, but, rather, extend the existing AOT cache creation workflow. Ensure that shifting from AOT-compiled code to JIT-compiled code is invisible to applications. Support the AArch64 and x64 processor architectures. Non-Goals It is not a goal to provide an AOT-only mode. Applications will use both AOT-compiled code and JIT-compiled code in the same run, automatically transitioning between them as needed. It is not a goal to support cross-compilation. Code compiled in a training run must run on the same CPU architecture, with the same set of features, in subsequent production runs. It is not a goal to support all CPU architectures currently supported by HotSpot. We expect normal porting activities to eventually add support for all major architectures. Motivation When a Java application is run by the HotSpot JVM, it passes through three phases: It starts up, then it warms up, and then it reaches peak performance. During startup, HotSpot invokes the application's main method and loads, links, and initializes classes on demand. Initially, it runs both application and JDK library code via the bytecode interpreter, which is slow. Inside the interpreter, HotSpot profiles the application's behavior by counting events such as method invocations and loop iterations. It uses the profile data to select frequently-invoked methods, or hot spots, which it compiles to native code via the basic C1 compiler. This native code is only modestly optimized. During warmup, the application settles into its workload and the loading, linking, and initialization of classes ramps down. HotSpot continues to profile the application, both in the bytecode interpreter and via instrumentation code inserted by C1. It collects richer profile information, including not just method-invocation and loop-iteration counts but also the types of objects encountered. As the profile data accumulates over time, it becomes more statistically useful. Eventually, HotSpot uses the data to select the hottest methods, which it compiles to native code via the advanced C2 compiler. This native code contains no instrumentation and is highly optimized. Profiling the application and generating native code is not free. Not only is the bytecode interpreter slow, but instrumented native code is slower than uninstrumented native code. Compiling methods to native code requires both CPU time and memory that could otherwise be used by the application, even though HotSpot only compiles methods to native code when profile data indicates that doing so will be worth the effort. Gradually, however, JIT compilation catches up to the application's emerging hot spots and the application runs faster. Eventually, all hot methods are compiled to fully optimized native code and the compilers go idle. The application remains in this state of peak performance as long as its hot spots do not change. The application's hot spots may change, however, in response to changes in its workload. When that happens, HotSpot can dynamically deoptimize, by discarding previously generated native code as needed, and reoptimize, by generating new native code for newly-hot methods. For example, if an application initially receives two types of requests then HotSpot dynamically optimizes the code for those two request types. If the application starts receiving a third type of request, HotSpot can dynamically deoptimize and then reoptimize the code for all three types of requests. The application can pass, in effect, through another warmup phase, sustaining performance as the application's workload changes. What about static compilation? Static compilation has sometimes been proposed as an alternative to the dynamic compilation of Java code. A static compiler converts entire applications to native code ahead-of-time, prior to run time. Static compilation has some advantages over dynamic compilation. A statically compiled application starts up and reaches peak performance immediately, without a warmup phase. At run time there is no need for a bytecode interpreter, profiling, or compilation. Peak performance can even be competitive with HotSpot if the static compiler's optimization work is guided by accurate profiles gathered during prior runs. Dynamic compilation, however, has three key advantages over static compilation. First, dynamic compilation makes applications agile because it responds to changes in the application's hot spots. It deoptimizes and reoptimizes as needed, sustaining performance as the application's workload changes. A statically compiled application cannot respond in this way — by its nature, it can be optimized for only one set of hot spots. Second, dynamic compilation makes applications portable across varying hardware and software because it generates native code at run time that is specific to the run-time environment. If an application is redeployed on a different processor architecture, a processor with a different feature set, a different operating system, or a different version of the JDK, HotSpot will achieve peak performance for that environment without requiring any change to the application. A statically compiled application must be recompiled in the face of such changes. Finally, dynamic compilation is compatible with the dynamic nature of the Java Platform. Features such as dynamic class loading, dynamic linkage, dynamic dispatch, and dynamic reflection bring vast expressive power, and have been fundamental to the platform's success. HotSpot handles these features naturally, while static compilers struggle with them. Even heroic amounts of static analysis cannot make up for the fact that these features require many decisions to be made at run time. Implementors of static compilers for Java code have therefore resorted to incompatible constraints, such as closed-world assumptions, and to putting significant burdens on developers, such as having to identify in advance the classes eligible for reflection. Shifting compilation work to training runs Throughout the startup and warmup phases, HotSpot continuously juggles multiple balls: It runs application and JDK library code; it loads, links, and initializes classes on demand; it profiles the application's execution; and it compiles hot methods to native code with varying degrees of optimization, guided by the profile data. The thesis of Project Leyden is that the key to improving startup and warmup time is to do some of this work earlier, ahead of time, rather than just in time. We shift work earlier in time by doing it in a training run, storing the results of the work in the AOT cache for instant use in subsequent production runs. We shifted class loading and linking work earlier in time via JEP 483, delivered in JDK 24. The AOT cache stores the loaded and linked forms of classes from the training run, thereby improving startup time. We shifted profiling work earlier in time via JEP 515, delivered in JDK 25. The AOT cache stores the execution profiles of methods invoked in the training run, enabling the C2 compiler to run immediately at the start of production runs, thereby improving warmup time. These improvements laid the foundation for our ultimate goal, which is to shift compilation and optimization work earlier in time. The AOT cache will store optimized native code compiled in the training run, enabling HotSpot to load that code instantly, rather than having to recompile it at the start of each production run. This will improve both startup and warmup time. HotSpot will not always use the cached code; if the application's workload changes then HotSpot can, as usual, deoptimize and reoptimize, generating new native code for newly-hot methods in order to sustain performance. Thus Java applications will gain some of the benefits of static compilation while retaining the agility, portability, and compatibility of dynamic compilation. Description We extend the existing AOT cache to store optimized native code generated in a training run. Such cached code is known as AOT code. During a production run, a request for optimized code for a method can be fulfilled instantly if matching AOT code is found in the cache. If AOT code is unavailable, incompatible, otherwise unsuitable, or later deoptimized, execution falls back to the existing interpreter and JIT mechanisms. AOT code and JIT code can coexist and are completely interoperable since they are created by the same compilers, C1 and C2. To create an AOT cache, use the AOTCacheOutput option to do a training run of your application and generate AOT code: $ java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ... This workflow is unchanged from previous releases. The AOT cache in the file app.aot, however, now contains not just pre-linked classes and profiling data but also AOT code for selected hot methods. Subsequently, in production, you can run the application with the cache: $ java -XX:AOTCache=app.aot -cp app.jar com.example.App ...