WIP/RFC: Unify all our various IR data structures into one by Keno · Pull Request #62334 · JuliaLang/julia

and others added 5 commits July 11, 2026 00:55 UnifiedIR is one IR data structure intended to span the compiler pipeline and external compilers (design specification in UnifiedIR/docs/design.md): a flat statement table with hybrid regions over a shared storage core (AttrGraph: kind column + packed two-mode operand words + one tagged operand pool + open attribute-column universes), layout states (builder/dense/editable/floating) with exactly two renaming points, a namespaced kind registry with statically reserved dialect ids for the bootstrap stack, a generic tree porcelain (Tree/NodeList cursors, construction, mapchildren/copy_ast, provenance walks over graph-qualified :source chains), compact!-as-GC (compact_graph!/collect_syntax!), verifier, printer/parser, and a reference interpreter. Zero dependencies. Wiring: UnifiedIR joins TOP_LEVEL_PKGS (symlinked to the julia share directory), is bootstrapped into Base immediately before JuliaSyntax (base/Base.jl) — JuliaSyntax's SyntaxGraph runs on this substrate — and its sources join COMPILER_FRONTEND_SRCS so sysbase rebuilds on change. The Base-baked instance keeps its kind registry clean of the test dialect (registration is gated to package mode). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> One storage core: SyntaxGraph wraps UnifiedIR.AttrGraph — kind is a real core column (the :kind attribute is a Dict-shaped view of it), children are STMT-tagged words in the shared operand pool addressed by packed range words, and the historical edge_ranges/edges/attributes properties are preserved exactly (identity-stable views; is_compatible_graph semantics kept). compact_graph! gives lowering the GC over syntax graphs it always wanted.

One porcelain: SyntaxTree{Attrs} = UnifiedIR.Tree{SyntaxGraph{Attrs}} and SyntaxList = UnifiedIR.NodeList — child indexing, attribute properties, newnode/newleaf/mknode/mkleaf/mktree, copy_attrs!/mapchildren/copy_ast, provenance walks, structural isapprox and printing are UnifiedIR generics; JuliaSyntax keeps only the genuinely syntax-specific parts (the Kind registry wrapper, SourceRef/source-text machinery, the leaf payload convention, parser integration, prune/unalias/@stm). One kind registry: the historical module-id mechanism re-plumbs into the shared UnifiedIR registry through register_kinds! — kind modules become dialects claiming contiguous opcode blocks (range predicates and BEGIN_/END_ markers preserved), the bootstrap stack claims statically reserved dialect ids (JuliaSyntax=1, JuliaLowering=2, formatter=3; core=0) so K"..." literals stay compile-time constants, and syntax K"call" and core K"call" are distinct kinds sharing one numbering space. Kinds re-register in __init__ (the registry is UnifiedIR session state); numbering is deterministic, keeping baked constants valid. Dual-mode: bootstrapped into Base it binds Base.UnifiedIR (included just before it); as a package it depends on the UnifiedIR package. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> JuliaLowering.UnifiedBackend (additive; zero changes to existing sources): reuses the front half unchanged — macro expansion, desugaring, scope analysis, closure conversion — and emits structured UnifiedIR region form directly (if/loop/try region ops, cells for frame variables, sealed exit terminators) instead of goto linear IR, with graph-qualified provenance: every emitted statement carries a :source column entry holding the originating syntax tree cursor, so the generic provenance walk crosses from optimized IR statements back to surface text (highlightable diagnostics), and collect_syntax! can GC the lowering graph against live IR. Pure definitions; nothing calls it during bootstrap, and it bakes cleanly in the optional sys-JL sysimage stage. Bindings resolve through the enclosing JuliaLowering (same JuliaSyntax and hence same UnifiedIR instance in both Base-baked and package modes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> …on UnifiedIR The provider side of the substrate (package-mode ONLY; the sysimage bootstrap never sees it): CodeInfo <-> UnifiedIR boundary converters (cfg-wrap entry incl. exception handlers; goto/typed exits with phi synthesis), an inference port running natively on UnifiedIR (differentially validated at 100% equal-or-better return types against the stock pipeline on session MethodInstances), optimizer passes (SROA, inlining via splice_body!, ADCE, structurization, cell promotion), the Queries API, and activation through the ordinary compiler-replacement mechanism (Core.OptimizedGenerics.CompilerPlugins.typeinf owner hook + with_unified_compiler), with every cached result round-tripped through a verified UnifiedIR shadow. Loading: Compiler.load_unified!() includes the port on demand into a Main-rooted carrier module and binds it as Compiler.Unified. On demand because the stdlib stub fast-path bypasses module evaluation when the package matches the sysimage copy; Main-rooted because this baremodule rebinds getproperty to raw getfield for its nested tree, which would break property-forwarding types. The plugin overload pins invoke_in_world to an end-of-module sentinel world so the shadow hooks are visible under both pkgimage and interpreted loading. Tests in Compiler/test/unified/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> - UnifiedIR/test wired into the test harness (choosetests + the runtests_vendored project-activation pattern of the other top-level packages): substrate, porcelain, kind registry, provenance walk, compact!/collect_syntax! GC, acceptance/fuzz/splice suites. - UnifiedIR/demo/provenance_demo.jl — the one-stack demonstration under the built binary: parse -> JuliaLowering front half -> UnifiedBackend.lower_to_ir (graph-qualified :source cursors) -> Compiler.Unified inference + UnifiedIR optimization -> highlight() of the exact surface text an optimized statement came from -> print_ir with per-statement source excerpts -> collect_syntax! (conservative and prune policies) -> the same highlight, byte-identical after AST GC. - NEWS.md entry under compiler/runtime improvements.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Keno mentioned this pull request Jul 11, 2026 Draft typed_ir returned the IR handle but its REPL display was the compact one-liner, which is useless for exploration. IR now shows as the full print_ir listing under MIME text/plain by default; truncation is opt-in, globally via display_maxlines!(n) or per-stream via IOContext(io, :ir_maxlines => n), with a tail note. Core.Const lattice types render as Const(value) instead of the opaque lattice fallback. @code_unified is the @code_typed-shaped entry point: argument values or bare ::T annotations, optimize=false for inference-only IR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Keno and others added 20 commits July 11, 2026 03:36 In Julia, `yield` reads as task suspension (Base.yield), and the name should stay available for genuine suspension points in the eventual await/continuation form (design doc §5.6). The region-value terminator is now `result`, pairing with `region_arg`: regions take region_args and feed their owner through result terminators. Mechanical rename across the core dialect, printer/parser, converters, optimizer, the lowering backend, and the design document; no semantic change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Add promote_arm_cells!, the if-join leg of §6 cell promotion, by store sinking: conditional stores in sibling arms become per-arm result values plus one unconditional post-join cell_set (extract-indexed when several cells tuple through one if), with the incoming value materialized as a cell_get before the if under definite assignment. Diverging arms contribute nothing; handler-observable, maybe-undef and token cells refuse per the §6 inviolables. In-arm reads reached by arm stores are rewritten through the reaching store, re-read at transform time so the rewrite is order-invariant across cells. optimize_ir! now runs arm and loop promotion to a joint fixpoint: arm sinking produces exactly the unconditional store shapes the dominating and loop passes consume, so each round can expose new cases for the others.

This makes the conditional-swap shape in Base.gcd (a, b = b, a under an if) promote fully: typed_ir(gcd, (Int,Int)) retains zero cell ops, and the corpus optimizer wall drops ~18% (the earlier promotion shrinks the IR for every later round). The passes record their join placements in PROMOTION_TRACE when the completeness harness asks for them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Add the fourth §6 join-point class: liveness-pruned, IDF-placed SSA construction for frame cells over cfg-op block graphs. Phis are block region args; incoming values ride the edge bundles of block terminators and of sealed exits of nested islands that land on our blocks mid-block (§5.5) — a mid-block edge exports the reaching definition at the sealed sub-op's member position, which stays well-defined through handler- nested exits because candidate stores are all direct block members and cannot execute mid-try. The incoming value is a cell_get inserted before the cfg op under definite assignment, folded by the dominating pass next round. Entry-leading cell_new (NewvarNode form) is honored by invalidating the incoming value. Soundness rule found by the DF harness: when a loop body lies strictly between the cfg op and the cell declaration, cell memory is carried ACROSS iterations through the island's sealed continue/break exits, so deleting island stores would leave the next iteration's incoming read stale — such cells refuse unless the island is store-free for them. Dissolving that class needs exit-value threading through sealed exits (deferred; the dominant remaining stock-oracle exception). promote_arm_cells! now also fires inside island blocks — its transform is local to the if's region subtree, and its post-join store is exactly what the island and block passes consume — and optimize_ir!'s promotion fixpoint includes the island pass. Stock-oracle violations on the Base/stdlib corpus drop from 38 bodies to 16, all in the documented exit-threading class; corpus cell-op elimination rises from 20% to 55%.