GEP-27

Groovy compiles each closure literal to a distinct generated inner class extending groovy.lang.Closure. That class carries a doCall method, a serialVersionUID, its own metaclass initialiser, a constructor taking (owner, thisObject, …​captured), and one Reference-boxed field per captured mutable local. Nesting concatenates names:

compiles to four classes — Deep, Deep$_render_closure1, Deep$_render_closure1$_closure2, and Deep$_render_closure1$_closure2$_closure3 — one per nesting level, names growing with depth.

The cost is not correctness but packaging: class-file bytes, classloading, bytecode verification, JIT profiling, and metaspace, all multiplied by the number of closures. For closure-heavy code — GSP page compilation, builders, DSL-shaped code — this is a measurable tax paid on code that reads as a one-liner.

Even for a statically compiled lambda that targets a functional interface, Groovy generates a full groovy.lang.Closure subclass:

Both generated classes extend Closure and implement GeneratedLambda, each with its own metaclass machinery. Groovy already uses invokedynamic/LambdaMetafactory (so the runtime value is a JDK metafactory Function, and the non-capturing case is already a zero-allocation singleton) — but it still drags along a per-lambda class whose only live part is a static doCall. The equivalent Java lambda emits no class per lambda: the body is a private static synthetic method on the enclosing class and the metafactory bootstraps directly against it.

The primitive that closes this gap for lambdas — "put the body on the enclosing class" — is the same primitive that fixes the closure explosion. That shared primitive is the heart of this GEP.

The question that selects the representation is what must this value be able to do at its use sites? Crucially, this is a short list. A shared Closure adapter that dispatches to a hoisted method is still a real Closure, so it preserves almost everything: it is callable, it has per-instance identity, and curry/rcurry/ncurry, memoize*, trampoline, compose, and iteration all continue to work (they operate through call()).

A hoisted representation loses exactly three things relative to a normal generated closure:

So "needs closure identity" is not a broad lattice; it is essentially the predicate "does externally-set delegate/resolveStrategy or serialization reach this value?"

Compilation mode is not a strategy selector; it is an information and dispatch modifier that flips which half of the problem is hard:

So the difficulty is not "dynamic vs static" but "free name vs not". The no-free-name subset — a large and common class (comparators, mappers, predicates over their own parameters, e.g. { x → x + 1 } or { a, b → a <⇒ b }) — is provably safe in either mode with no types. The free-name subset is where static compilation’s proof earns its keep, and where dynamic compilation must fall back to the trust declaration.

Given the required capability, the compiler picks the lightest representation that satisfies it:

Capturing modulates each (non-capturing enables the singleton/static-method form); static mode enables typed dispatch of the hoisted body and the S2 path.

Under @CompileStatic the escape analysis proves delegate-independence and declines the rest, so the boundary is enforced by construction. Under dynamic compilation @PackedClosures is a trust assertion the compiler cannot verify — so the design must protect a user who annotates in good faith and gets it wrong. Two mechanisms do this, and they deliberately cover different failure modes:

A conservative escape gate shrinks the runtime surface further and is implemented in the S1 slice: a closure literal that visibly escapes its method — stored into a field/property/index (the Grails static constraints = { … } shape), initialising a field (def action = { … } at class level, which compiles outside the enclosing method’s visible code and is checked against the class’s fields directly), returned, appended with <<, or placed in a collection literal — is declined at compile time and kept as a class. This moves the clearest delegate-risk cases from a runtime throw to a compile-time non-event; those declines are also what WARN/STRICT report. A sibling gate applies the same discipline to the serialization boundary: a literal that is visibly serialization-bound — cast or coerced to a Serializable type, passed directly to a writeObject call, or held in a local that is (the canonical def c = { … }; out.writeObject(c) shape, tracked one def-use hop within the method) — declines and keeps its class (so it serializes as before), with genuinely transitive routes (helper methods, constructor-mediated graphs) left to the adapter’s fail-fast writeObject, exactly as transitive escapes are left to the delegate guard. Because syntactic escape analysis cannot prove that, say, a custom each does not set a delegate, the runtime guard remains the essential backstop under dynamic compilation.

The delegate concern only arises for a closure that resolves a free name. A closure whose body contains none — no implicit-this method call, no unqualified/dynamic variable, only its parameters, locals, captures, constants and explicit/parameter receivers — cannot be reached by any delegate, so it is delegate-independent by construction, with no type information at all. This is a large and everyday class: { x → x + 1 }, { it * 2 }, { a, b → a <⇒ b }, { it.name }, { x → x + captured }. Such closures are sound to pack under dynamic compilation too, without @PackedClosures and without a runtime guard (a caller-set delegate is a harmless no-op, exactly as on a normal closure), because there is nothing to prove — a purely syntactic check of the body decides it.

This is implemented: with the flag on and no annotation, the closures above auto-pack under dynamic compilation and behave identically (including a set delegate being ignored, not rejected), while a closure with a free name ({ owned() + x }, an implicit-this call, or { x + missing }, a dynamic variable) correctly stays a class. Because a dynamically-compiled lambda is itself compiled as a closure, the same syntactic path also body-hoists the equivalent dynamic lambda subset (e.g. (x) → x + 1) for free — it just does not reach the LambdaMetafactory singleton form, which remains @CompileStatic-only for the mechanical reason that the metafactory needs the target functional-interface type.

This makes the default-on story broader than "the @CompileStatic-proven set": a syntactic delegate-independence check widens automatic packing to the provably-safe dynamic subset as well. Free-name dynamic closures remain opt-in (@PackedClosures + guard), and nested closures still await owner-correct retargeting (Groovy 7) — but the trivial subset need not wait for either.

(A closure that is fully self-contained is also safe to pack even where it escapes — an escaping { x → x + 1 } cannot misbehave — but exploiting that for nested closures needs owner-correct retargeting, see Reference implementation.)

The complete boundary, as implemented (ClosureWriter#chooseStrategy; any decline keeps the closure a generated class exactly as today, and is what WARN/STRICT — or groovy.target.closure.pack.report=true on the flag path — reports with a reason):

Everything that passes packs. The matrix is pinned as executable documentation by PackedClosureBoundariesTest (18 shapes × both compilation modes, plus the special-context shapes), and each gate has focused behavioural tests besides.

Both pieces below are merged (GROOVY-12143), behind the opt-in groovy.target.lambda.hoist flag; the mutated-capture and serializable cases correctly stay on the class-based path.

For a statically compiled, non-capturing, non-serializable lambda targeting a functional interface, do what Java does: emit the body as a private static synthetic method on the enclosing class and bootstrap LambdaMetafactory directly against it — generating no lambda class.

This is sound by construction: the target is a SAM (no delegate possible), the runtime value is already a metafactory instance (so the lambda class was never the value — it was a dead impl-holder), and non-capturing means no capture threading. Measured on a proof-of-concept:

The metafactory singleton is preserved (nonCap().is(nonCap()) == true), stack traces name a $lambda$…​ method on the enclosing class (Java-like), and behaviour is unchanged across Predicate/BiFunction, lambdas in static methods, lambdas calling static methods, and serializable non-capturing round-trips (which fall back to the class-based path). A lambda that reads an instance field (() → count) is correctly not hoisted and keeps working.

The change is localised to StaticTypesLambdaWriter (the existing doCall construction already does the correct non-capturing/instance-member analysis; the hoist re-homes the resulting static method onto the enclosing class). It requires no change to the invokedynamic/metafactory bootstrap other than the implementation method’s owner class, and does not require the class-visitor changes the closure work needs.

A capturing SAM lambda today instantiates its lambda class to hold the captured values (Groovy even Reference-wraps them). The Java-style form uses a static implementation method taking the captured values as leading arguments, with the metafactory capturing them directly — which also removes the Closure allocation and the Reference wrapping. This is merged for the read-only capture case (the common one): captured values are passed as typed leading parameters. A lambda that mutates a capture still needs the shared Reference, so it correctly declines and keeps its class. This lands the bulk of the capturing benefit in Groovy 6 behind the same flag; only mutated-capture lambdas remain on the class-based path.

A shared Closure adapter family (PackedClosure, instantiated as the fixed-arity member matching the literal — Fixed0..Fixed4, FixedIt for implicit-parameter literals, FixedN for higher and vararg arities) that dispatches to a hoisted body, applied to closures rather than lambdas — now implemented in the code generator (GROOVY-12151) and proposed for Groovy 6.0 as a stretch goal. It covers the hard cases: no-capture and captured closures (by value; written captures via shared Reference, including =`/`+), arbitrary-depth nested closures, implicit it, under both @CompileStatic and dynamic compilation, correctly declining closures that use delegate/owner/super, or that carry default parameter values (multi-arity — see Excluded and deferred features). All three safety mechanisms from Guarding the dynamic trust assertion are implemented and validated: the mandatory PackedClosure runtime guard, the compile-time escape decline, and the @PackedClosures(mode = WARN | STRICT) decline diagnostics. The two headline capabilities are done:

The API surface (@PackedClosures, PackMode) is marked @Incubating. What keeps this a stretch goal rather than a committed 6.0 deliverable is that it is opt-in only (the automatic path is not yet default-on) and the nested-closure, elision, and writer-unification work below is still open — all Groovy 7.0.

Today a self-contained nested closure (e.g. a Grails validator: { … } block inside constraints) is kept as a class, because hoisting it naively adds the body to the enclosing closure class with the wrong owner (see Grails artefact DSLs and nested closures). Hoisting it soundly requires retargeting the method onto the correct outermost owner, rebinding its free names to that owner, and emitting static-vs-instance dispatch to match. This is the single highest-value target — domain classes generate the most closure classes — and the reason it is a separate, later phase rather than a cheap gate change.

For closures/lambdas that neither escape nor need an object — for example immediately-invoked bodies — elide the object entirely. This is the most aggressive step and depends on escape proof and, for iteration intrinsics, knowledge of the callee.

Fold the lambda and closure code paths behind the single hoist backend, so the SAM and Closure cases are one mechanism selecting different wrappers.

Convergence also runs the other way: generated closure classes (S0 — everything packing declines) can adopt the packed dispatch discipline without being packed. The compiler knows every doCall signature at emission time, so it can emit a call(Object…​) override on the closure class that arity-switches directly to the right doCall (with the MOP-transparency guard inlined). The Java/GDK path then becomes plain virtual dispatch — capability-clean under JPMS, no reflection — and the runtime’s reflective doCall cache in Closure.call demotes to serving only legacy jars and hand-written subclasses. That cache is a bridge, not a destination.

A related follow-up surfaced in review: MethodClosure (from .& method references) is the same shape — a Closure whose target is a known method that cannot honour a delegate — but it dispatches by name with runtime overload selection and silently ignores a caller-set delegate, whereas PackedClosure dispatches through a fixed table entry and rejects one. The two are worth contract-aligning (a shared "known-target, delegate-inert" marker and consistent delegate handling), and statically resolved method pointers could claim ids in the same per-class dispatch tables the packed adapter already uses — dispatching with the host class’s own bytecode rights, which also gives caller-sensitivity-correct semantics. Their dispatch models differ enough, though — a fixed table entry versus runtime overload selection — that fully merging them into one path is not a goal; this is tracked as a separate initiative.

Flip closure packing from experimental (opt-in groovy.target.closure.pack) to default-on. This is a policy decision gated on the remaining human-only tooling verification (see IDE and tooling considerations) — not on further mechanism work: the dispatch and MOP-transparency mechanisms are complete and verified (interception, categories, and introspection behave identically to generated closure classes on every path), and the remaining documented residuals are those listed under The one that matters: required capability.

The prototype hoisted method carries the debug metadata a debugger relies on, verified both by inspecting javap -p -l output and by an automated test (LambdaHoistTest) that reads the class bytes with ASM and asserts the following on every hoisted $lambda$…​ method:

Because the hoisted body reuses the original AST statements, source positions propagate unchanged; there is no separate line-mapping step that could drift.

The packed-closure leg has the equivalent automated coverage (PackedClosureDebugMetadataTest): line-number entries per body line and named locals — including captured values (a written capture threaded as a shared Reference still appears under its original name), typed and implicit-it parameters, and body locals — on the $packed$closure$ methods under both compilation modes, plus ACC_SYNTHETIC on the hoisted bodies and on all generated dispatch machinery.

The following must be exercised interactively in the reference IDEs and cannot be asserted from a build, so they remain open items before the flag defaults on:

Closure classes are dominated by scaffolding, not logic. On a class reproducing seven real FormTagLib closure patterns, packing eliminated all seven closure classes (8 class files → 1; 23.8 KB → 7.3 KB, ~69% fewer bytes); the hoisted method bodies added only 1.8 KB, so roughly 90% of a closure class’s bytes were Closure scaffolding — constructor, metaclass init, serialVersionUID, Reference fields — rather than the closure’s own code. Grails domain classes are the extreme case: a representative RemarkCriteria domain generated 24 closure classes totalling 80 KB — more bytecode than the entire rest of the class (1.19×).

Dispatch speed matters as much as class count, because a shared adapter is invoked through the Closure machinery rather than being its own class. A naive adapter (reflective invoke plus per-call argument adaptation) measures ~7–9× slower to call() than the generated closure class it replaces — smaller but slower — and a per-instance MethodHandle fares little better (the JIT cannot constant-fold it, so every call runs the method-handle invoker). The shipped design avoids both: the per-class dispatch tables execute as ordinary bytecode the JIT inlines through, the per-arity tables pass arguments as plain parameters (no array to allocate or escape), and constant-dynamic parameter-type arrays remove the last per-creation allocation. Measured with JMH on the same build, capturing closures — the shapes GEP-27 exists for — run 1.4–1.9× faster packed than as generated classes, holding under megamorphic dispatch; the tightest non-capturing collect trails at ~0.6×, the cost being the MOP-transparency guard (loop-invariant checks the JIT hoists when the chain inlines — the same checks the classic call-site caches make). On Grails-shaped in-situ workloads the blend nets +17–23% packed with identical outputs, so the adverse micro-shape is real but dilutes out under real work (further lane micro-tuning was assessed against those shapes and found to have no in-situ effect). Ratios are measured against a current baseline that includes the Closure.call fast-path improvements (GROOVY-12164/12165), i.e. against generated classes at their fastest.

The delegate DSLs Grails relies on — static constraints = { … }, static mapping, namedQueries, old-style def action = { … } — are field-assigned, which is exactly the escape gate’s decline signal, so they are kept as real closure classes and their delegate resolution is unchanged (verified end-to-end against a builder delegate). Because Grails discovers these blocks by their being fields, the whole family lands on the safe side by construction rather than by special-casing; and the Groovy 6 SAM-lambda work never touches them at all (they are `Closure`s, not lambdas).

The value left on the table in a domain class is the many self-contained nested closures — the validator: { val, obj → … } blocks inside constraints, which reference only their own parameters and owner statics, not the constraints delegate. A spike that relaxed the nested-closure decline for these confirmed the class-count prize (the shown RemarkCriteria subset dropped 6 → 2 closure classes) but proved the naive form unsound: the hoisted body was added to the enclosing closure class as an instance method, while a static constraints block’s owner resolves to the domain class, so dispatch attempted a non-existent static method (MissingMethodException), and where it did dispatch, owner-static references resolved against the wrong owner. Hoisting a nested closure therefore requires retargeting the method onto the correct outermost owner, rebinding its free names to that owner, and emitting static-vs-instance dispatch to match — the owner-correct capability work already scoped to Groovy 7. This makes domain classes the single highest-value target for that phase, and the reason it is Groovy 7 and not a cheap gate change.

A supporting change is worth noting: hoisting nested closures requires the class visitor to visit methods added during code generation to a fixpoint (a two-round visitMethods misses methods added while compiling a method that was itself added during compilation). That change is implemented in the packing slice — an identity-based fixpoint, also fixing an O(n²) scan against method-generating visitors — and is validated against the full suite.

Phases 1–2 stand alone and deliver value with no dependency on the later phases. Phase 3 (the stretch goal) builds on the shared backend proven in phases 1–2; phases 4–5 build on phase 3.

The feature is additive and reversible:

Generated-class names and the presence of per-closure/per-lambda classes are not part of any public API, but tooling may depend on them (see IDE and tooling considerations). The flag is the compatibility valve while tools are updated.

No new trust boundary. The optimisation is a local codegen transform; it introduces no new runtime input handling.