JIT compilation lets the JVM observe runtime behavior and compile frequently executed bytecode into optimized machine code during the life of the process.
Learning Question
Why can Java code get faster after it has already started running?
Java compilation has two different meanings that are easy to mix together.
javac compiles Java source into class files before runtime. The JIT compiler compiles bytecode into machine code during runtime.
The first mental model is:
javacproduces portable bytecode. The JIT compiler produces process-local optimized machine code.
Interpreter and Compiler
When a method first runs, the JVM may interpret its bytecode or use already compiled code depending on runtime state and JVM configuration.
Interpreting means a JVM interpreter executes the bytecode instruction by instruction through native interpreter code. This starts quickly and allows the JVM to gather information.
If a method or loop becomes hot, the JVM may compile it into machine code. The CPU can then execute that generated machine code directly.
This is one reason Java programs often have warmup behavior. Early execution and later execution may not have the same performance characteristics.
Runtime Profiling
The JVM can observe runtime behavior while code executes.
Useful profile information can include:
- which methods and loops are hot
- which concrete classes appear at virtual call sites
- which branches are commonly taken
- whether allocations appear to escape a method
- whether locks are frequently contended
The JIT compiler uses this information to make better optimization decisions than a purely ahead-of-time compiler might make without runtime knowledge.
Common Optimizations
JIT optimizations can include:
- inlining small or hot methods
- removing virtual dispatch when the receiver type is predictable
- eliminating bounds checks when safety can be proven
- removing or scalar-replacing allocations that do not escape
- folding constants and simplifying branches
- optimizing loops
- removing synchronization when it is provably unnecessary
These optimizations must preserve Java-visible behavior. If runtime assumptions become invalid, the JVM can deoptimize and return execution to a less optimized form.
Speculation and Deoptimization
The JVM may make speculative assumptions based on observed behavior.
For example, a virtual call site may only have seen one implementation so far. The JIT may optimize for that case. If a new class is later loaded or a different receiver type appears, the assumption can be invalidated.
Deoptimization is the process of abandoning optimized machine code and reconstructing a valid interpreter-level or less optimized execution state.
This is one reason the JVM needs metadata and safepoint coordination even after code has been compiled.
Warmup
Warmup is the period where a JVM process loads classes, initializes frameworks, fills caches, interprets code, gathers profiles, and compiles hot paths.
During warmup:
- more classes may be loaded
- static initialization may run
- caches may populate
- JIT compilation may occur
- GC behavior may differ from steady state
- branch and type profiles may stabilize
Benchmarking only cold execution can be misleading for long-running services. Benchmarking only warmed-up execution can also hide startup costs that matter for short-lived processes.
The Code Cache
JIT-compiled machine code lives in memory managed by the JVM, commonly discussed as the code cache in HotSpot.
Code cache pressure can affect performance because the JVM needs space to store generated code. This is another example of JVM memory that is not ordinary Java heap.
Core Mental Model
Keep these boundaries separate:
javaccompiles source into class files before runtime.- The interpreter executes bytecode through JVM native code.
- The JIT compiler turns hot bytecode into machine code during runtime.
- Runtime profiling lets the JVM optimize based on observed behavior.
- Optimized code can be invalidated and deoptimized when assumptions fail.
- Warmup is real runtime work, not just application business logic running slowly.
Final Summary
JIT compilation is why Java execution is dynamic even after compilation.
A JVM can start from portable bytecode, observe the running program, generate optimized machine code for hot paths, and adapt when runtime assumptions change.