Skip to main content

Adesh Compiler Architecture

● PARTIAL(Compiler stages and backend feature coverage vary; the native ADOB toolchain is partial)

The Adesh compiler has a shared front end and dedicated execution paths. The interpreter, JIT compiler, and WASM paths consume low-level IR. The native AOT compilation pipeline is self-contained: it lowers HIR into Machine IR, emits validated ADOB v3 objects, and links with adeshlink into native PE, ELF, and Mach-O binaries without any LLVM or external compiler/linker dependencies.


Full Pipeline ASCII Diagram​

.adesh source text
│
▼
┌─────────────┐ src/parsing/lexer.rs
│ Lexer │──── token stream (Token { kind, lexeme, line, col, line_text })
└──────┬──────┘ src/parsing/error.rs (rich diagnostics)
│
▼
┌─────────────┐ src/parsing/parser/*.rs
│ Parser │──── AST (src/parsing/ast.rs) — lossless, typed nodes
└──────┬──────┘ parser/core.rs, statements.rs, expressions*.rs,
│ type_annotations.rs, type_decls.rs, literals.rs,
│ functions.rs, classes.rs, extern_ffi.rs, declarations.rs
│
▼
┌──────────────────┐ src/parsing/ast_optimizer.rs
│ AST Optimizer │── constant folding on AST, node normalization
└──────┬───────────┘
│
▼
┌─────────────┐ src/ir/hir/*, src/parsing/hir_lower.rs, src/parsing/hir_passes.rs
│ HIR │──── desugared, resolved, explicit scopes
└──────┬──────┘ elif/ternary normalized, this/self unified
│
▼
┌──────────────────┐ src/semantics/engine.rs, src/semantics/mod.rs
│ Semantic Analysis│── name resolution, type inference & checking,
└──────┬───────────┘ generic bounds, trait resolution, control-flow legality
│
▼
┌──────────────────┐ src/parsing/cfg_borrow/*, src/parsing/borrow_check.rs,
│ Borrow Checker │ src/ir/mir/*, src/parsing/ownership*.rs
│ + MIR │── ownership graph, move analysis, ARC insertion,
└──────┬───────────┘ lifetime inference, drop insertion, CFG validation
│
▼
┌─────────────┐ src/ir/vir/* (instructions.rs, lower.rs, types.rs, validate.rs)
│ VIR │──── SSA-form low-level IR, unified input for all backends
└──────┬──────┘ src/ir/optimizations/* (constant_folding, CSE, DCE, inlining)
│ src/ir/simd/* (analysis, vectorize, cost_model, lowering)
│ src/ir/parallel/* (analysis, transform, cost_model)
│
▼
Execution backends
Interpreter / JIT Compiler / AOT Native / WASM
│
▼
Runtime-dependent execution

Self-contained native AOT path: HIR → Machine IR → x86-64 / AArch64 codegen → ADOB v3 → adeshlink → Native Executable.

Data-flow summary​

Source ─► Lexer ─► Parser ─► AST ─► HIR
.adesh tokens tree desugar ├──► Semantic ─► MIR/Borrow ─► VIR ─► Interpreter / JIT / WASM
└──► Native Lowering ─► Machine IR ─► ADOB v3 ─► adeshlink ─► Native Binary

The native toolchain work is planned for v0.4.0. Windows x86-64 is end-to-end tested; other linker and architecture targets have narrower or unverified support. See Native Compiler Toolchain.


Compiler Stages & Implementation Mapping​

#Pipeline StageResponsibilitiesSource Path
1LexerTokenization, BOM handling, operator lexing (??, ?., **, ~/, ..., =>), comment/doc-comment stripping, position trackingsrc/parsing/lexer.rs, src/parsing/error.rs
2ParserRecursive-descent grammar, expression/statement/type-annotation parsing, error recovery to next statement boundarysrc/parsing/parser — core.rs, statements.rs, expressions.rs, expressions_primary.rs, expressions_unary.rs, literals.rs, functions.rs, classes.rs, type_annotations.rs, type_decls.rs, declarations.rs, extern_ffi.rs
3ASTLossless typed tree (FnStmt, ClassStmt, StructStmt, EnumStmt, MatchExpr, decorators, generics, template literals), receiver normalizationsrc/parsing/ast.rs, src/parsing/ast_optimizer.rs
4HIRDesugaring (elif chains, ternary → canonical if), symbol resolution, scope explicitation, this/self unificationsrc/ir/hir — mod.rs, lower.rs; src/parsing/hir_lower.rs, src/parsing/hir_passes.rs, src/parsing/hir.rs
5Semantic AnalysisType inference, trait/generic bound checking, function signature validation, field access, control-flow legality, pattern-match exhaustivenesssrc/semantics — engine.rs, mod.rs; src/types, src/typesystem
6Borrow / Ownership (MIR)Ownership graph, move/borrow tracking, CFG lifetime validation, ARC insertion, drop insertion, escape analysissrc/parsing/cfg_borrow, src/ir/mir — mod.rs, lower.rs, types.rs, validate.rs, borrow_analysis.rs, ownership_graph.rs, arc_insertion.rs, drop_insertion.rs, lifetime_inference.rs; also src/parsing/borrow_check.rs, borrow_inference.rs, ownership.rs, ownership_enhanced.rs, lifetime_tracking.rs, escape_analysis.rs, variance.rs
7VIR & OptimizationsSSA lowering, constant folding/propagation, CSE, DCE, inlining, SIMD vectorization pass, parallel transform, validation & pretty-printingsrc/ir/vir — instructions.rs, lower.rs, types.rs, validate.rs, pretty_print.rs; src/ir/optimizations; src/ir/simd; src/ir/parallel
8CodeGen BackendsReference Interpreter, JIT Compiler, self-contained native AOT toolchain, and WASM emittersrc/backends/, crates/adesh-codegen/, crates/adesh-object/, linker/
9Runtime16-byte Value enum, string interning, shared Arc ownership, nested environments, promise microtask queue, timers, builtin registrysrc/execution/, src/typesystem/value_optimized.rs, src/utils/interner.rs

Core Invariants​

  1. Source of Truth Integrity: Semantic analysis runs on desugared HIR before VIR lowering. The checker never sees surface sugar.
  2. Compile-Time Safety: Borrow checking and move analysis reject illegal aliasing, use-after-move, and lifetime violations before code generation. See Borrow Checker.
  3. Backend boundaries: The interpreter serves as the language correctness baseline. The native AOT path lowers via Machine IR and ADOB v3 to adeshlink without LLVM or external toolchains.
  4. Diagnostics First: Every stage preserves source positions (line, col, line_text) so errors render with --> file:line:col and source context.
  5. GC-less Determinism: No garbage collector. Ownership, Arc, and RAII drops give deterministic memory behavior (see Memory Management).

How the Stages Interact​

┌─────────────────────────────────────────────┐
│ Frontend (parsing) │
source.adesh ─► Lexer ─► Parser ─► AST ─► ast_optimizer │
└──────────────────────┬──────────────────────┘
│
┌──────────────────────▼──────────────────────┐
│ Middle-end (analysis) │
│ HIR lower ─► Semantics ─► MIR/Borrow │
│ (scopes) (types) (ownership/CFG) │
└──────────────────────┬──────────────────────┘
│
┌──────────────────────▼──────────────────────┐
│ Low-end (IR + opt) │
│ VIR lower ─► optimizations ─► VIR validate │
│ (SSA) (fold/CSE/DCE/inline/SIMD) │
└──────────────────────┬──────────────────────┘
│
┌──────────────────────▼──────────────────────┐
│ Backends (codegen) │
│ Execution backends: Interpreter │ JIT Compiler │ AOT Native │ WASM │
│
▼
Runtime Value

The partial native path branches from HIR into Machine IR, then ADOB and adeshlink; it is documented separately from the VIR backend flow above.

  • Frontend is syntax-aware but type-blind; its job is to produce a well-formed tree.
  • Middle-end is where meaning is checked: names, types, ownership.
  • Low-end erases high-level constructs into flat VIR instructions and optimizes them.
  • Backends consume VIR and emit executable behavior.

Inspecting the Pipeline​

The CLI exposes IR dumps for learning and debugging:

adesh run --emit=hir --njit myapp.adesh # dump HIR (desugared, resolved)
adesh run --emit=mir --njit myapp.adesh # dump MIR (ownership-annotated)
adesh run --emit=vir --njit myapp.adesh # dump VIR (SSA + optimized)
adesh run --emit=cfg --njit myapp.adesh # dump CFG for borrow analysis
adesh --help # all flags defined in src/cli/args.rs

REPL helpers:

:ast let x = 42; // parser output
:type x // semantic type
:env // current environment / scopes

Inside the Pipeline — Deep Dives​

StageDeep dive
Source text → tokens → ASTLexer & Parser
HIR + type checking + symbol resolutionSemantic Analysis & HIR
Ownership, borrows, drops — compile-time safetyMemory Safety Analysis
HIR → MIR → VIR, optimization & SIMD passesThe IR Pipeline
VIR → interpreter / JIT / AOT / WASMCodegen & Backends
Runtime values, environments, async servicesRuntime & Value Model
Allocators, arenas, Arc, cycle detectionMemory Management

Design Rationale​

DecisionWhy
Recursive-descent parserPredictable, debuggable, excellent error recovery; one error doesn't hide the next
Lossless ASTPreserves source fidelity for diagnostics and tooling (formatters, linters)
HIR desugaring before semanticsChecker sees one canonical form — fewer special cases, fewer bugs
MIR for borrow checkingControl-flow graph makes lifetime/move errors decidable and explainable
VIR as single backend inputBackends stay consistent; new targets implement one interface
GC-less runtimeLatency-free, cache-friendly; ownership checked at compile time

Source Tree Quick Reference​

src/
├── parsing/ # Lexer, Parser, AST, HIR lower, borrow/ownership analyses
│ ├── lexer.rs
│ ├── ast.rs / ast_optimizer.rs
│ ├── hir.rs / hir_lower.rs / hir_passes.rs
│ ├── parser/*.rs # 12 sub-parsers
│ ├── cfg_borrow/ # CFG + borrow checker
│ └── borrow_check.rs, ownership.rs, lifetime_tracking.rs, ...
├── semantics/ # Type checking, symbol resolution
│ ├── engine.rs
│ └── mod.rs
├── types / typesystem/ # Type definitions + runtime Value
├── ir/
│ ├── hir/ # HIR definition & lowering
│ ├── mir/ # MIR + borrow/ownership analyses
│ ├── vir/ # VIR instructions, lowering, validation
│ ├── optimizations/ # Fold, propagate, CSE, DCE, inline
│ ├── simd/ # Auto-vectorization
│ └── parallel/ # Auto-parallelization
├── backends/
│ ├── interpreter_backend/
│ ├── jit/ # baseline, adaptive, cranelift, tiered
│ ├── aot/ # abi, cache, object_gen, static_linker
│ ├── wasm_backend/ / wasm/
│ ├── mlir/ # GPU path
│ ├── llvm/ # alternative native path
│ └── common/ffi
└── execution/ # Interpreter runtime, environments, async