Compiler qualification investigation

Status: investigation plan, 2026-10-17. This document does not qualify a compiler, runtime, or package change.

Investigate two separate capabilities:

1. Use tsc-ts to check TypeScript and emit the existing Node.js package faster.
2. Use scriptc to compile fs-safe application code into native programs while preserving its safety contracts.

The second capability changes the runtime. A faster compiler does not establish faster filesystem operations. Keep both investigations open until measurements and contract proof support a decision. Prepare small draft PRs. Obtain explicit maintainer approval before merging, releasing, or deploying anything.

#Frozen baseline

Do not equate a release binary with a later source revision that declares the same package version. Freeze source, lockfile, compiler libraries, runner, flags, and binary identities for each comparison. Changes to those inputs require new evidence for the affected experiment.

The frozen tsconfig includes 317 source TypeScript files. It excludes the test tree and JavaScript tooling. Its contract includes strict NodeNext, ES2022 output, declarations, noUncheckedIndexedAccess, and verbatimModuleSyntax.

The build script invokes the official compiler through Node. It then prepares the Windows helper and Rust/WASM archive artifact. Measure the compiler separately from the complete build.

The package API checker imports typescript/unstable/ast and typescript/unstable/sync. Keep that dependency when evaluating a different compiler CLI. Its export-name and error-code checks do not prove complete declaration-signature parity.

#Hypotheses and experiments

#Compiler parity protocol

1. Record source and lockfile hashes, effective config, resolved input files, compiler versions, libraries, and binary hashes.
2. Compare the locked official compiler with the candidate native executable.
3. Use the matching TypeScript-Go oracle and Node-hosted tsc-ts to localize disagreements when needed.
4. Compare diagnostics by category, code, relative file, span, and message.
5. Include valid consumers and intentional type errors. A candidate must not silently omit invalid inputs.
6. Compare emitted file inventories and bytes, including every .js and .d.ts file.
7. Investigate differences before introducing normalization. Preserve error locations and meaningful output differences.
8. Compile consumer fixtures against both declaration sets. Cover overloads, generics, unions, disposal, defaults, and public subpaths.

Reuse upstream corpus and package tests where they answer the same question. The inspected upstream benchmark needs care. Its whole-project timer ignores child status, and that comparison omits native tsc-ts. Record this as an upstream follow-up. Do not use that timing table as fs-safe qualification.

#First harness

Run scripts/compiler-qualification.mjs on an isolated validation host after pnpm install --frozen-lockfile. Supply the candidate's native executable, not its npm JavaScript launcher. Keep its installer provenance separately: release, source commit, npm integrity, platform package, and download origin.

node scripts/compiler-qualification.mjs \
  --candidate /path/to/native/tsc-ts \
  --output /path/to/new-evidence-directory

The output directory must be new and its parent must exist. The harness resolves the locked official typescript@7.0.2 and its matching platform package, then uses both native executables directly. It sets TSC_TS_LIB_DIR to the official platform package's libraries and verifies the actual resolved input bytes. The official package remains installed for the existing API checker. The harness clears inherited TSC_TS_*, GOGC, GOMEMLIMIT, GOMAXPROCS, and GODEBUG tuning before setting the library directory. It records cleared names.

The harness requires identical effective configs, source checks, one valid and one deliberately invalid language fixture, and emitted JS/declaration inventories and bytes before timing. It records CLI diagnostic start-location parity; CLI text does not expose diagnostic end positions or lengths. Structured-span and public declaration-consumer proof remain separate experiments.

report.json records hashes, versions, effective config, input/output inventories, process status and streams, and every successful timing sample. A failed process remains in the process log and makes the entire campaign fail. No failed campaign qualifies performance. Initial and final hashes reject changed source, inputs, compiler executables, libraries, manifests, and retained output.

Default timing is two warmups and ten measured runs per compiler for each of checking and emission, interleaved in ABBA/BAAB blocks. Counts must be positive even integers. --single-threaded passes the same flag to both compilers; unsupported flags fail the campaign. --timeout-ms defaults to 120000 per process. This first harness measures wall time only. CPU time and peak RSS remain unmeasured. Filesystem cache state is uncontrolled. It does not qualify the complete build, installed package, target runtime, or a compiler replacement.

Use official/ and candidate/ only as retained compiler-output artifacts. Successful timing samples discard their duplicate emit directories after hashing. The harness never writes dist or invokes a packaging command that rebuilds it. Review the report for private diagnostics before publishing it.

#Performance protocol

Use one isolated runner without competing builds. Record CPU allocation, operating system, filesystem, runtime, and resource limits. Compare default concurrency and equal checker counts. Report single-threaded results separately.

Start with two warmups and ten measured runs per configuration, with interleaved ABBA/BAAB order. Report median, spread, wall time, CPU time, and peak RSS. Exclude warmups from reported samples. Classify nonzero exits, signals, crashes, incomplete output, and timeouts as failures rather than fast samples. Use a reviewed finite deadline for each experiment and report censored runs.

Distinguish fresh-process runs from filesystem-cache-cold runs. Do not claim cold caches without controlling them. Measure incremental no-op and one-file edits only as optional scenarios because the current build does not enable incremental compilation. Measure repeated API/watch operation separately if adoption later includes a persistent compiler process.

Select a material benefit threshold after measuring the baseline and before evaluating candidate timings. The initial official-only pilot took about 0.37 seconds per check or emit on a Linux x64 c7a.8xlarge runner. For this investigation, require at least 20% median compiler-time savings and 10% whole-build savings before proposing optional selection. Report absolute savings and complete-check impact as well; meeting those thresholds does not replace compatibility or maintenance review. Prefer whole-build savings and runner cost over a microbenchmark ratio. Do not adopt a candidate that saves negligible time while adding substantial platform or maintenance cost.

#Package and runtime proof

Most runtime tests import source through Vitest. Those tests do not prove candidate-emitted JavaScript. The experiment must preserve the identity of the generated package through every check.

Run the ordinary baseline through the existing repository commands: Use a qualified validation host with frozen dependencies and the documented Rust/LLVM toolchain.

pnpm check
pnpm native:build
pnpm package:smoke
git diff --check

For candidate output, compare compiler artifacts before packaging. Prepare required native/WASM/helper artifacts through their existing owners. Then run node scripts/check-pack.mjs against the candidate's dist in its isolated validation checkout. This checker packs with --ignore-scripts, imports public subpaths, and runs installed-package smoke checks. It also compiles installed consumers for root overloads, JSON stores, lock callbacks, resource disposal, and discriminated results, with strict checking and exact optional properties. To check those same consumers with the candidate compiler as well:

node scripts/check-pack.mjs --candidate-compiler /path/to/native/tsc-ts

Both compilers use the official standard library and the same installed tarball. An unused @ts-expect-error control must fail before negative consumer cases are trusted. The fixtures are compile-only; the existing package smoke tests own runtime behavior.

Do not run pnpm pack:check or pnpm check over candidate output and claim that output survived. Those commands rebuild with the official compiler in the frozen baseline. Do not update test/public-api.json merely to accept compiler differences.

Reuse the existing security cohort plus these representative lifecycle tests:

• test/root-read-final-fence.test.ts
• test/atomic-restore-error-precedence.test.ts
• test/atomic-restoration-falsy-settlement.test.ts
• test/descriptor-close-ownership.test.ts
• test/root-create-native-integration.test.ts
• test/root-mkdir-post-create.test.ts
• test/root-write-lifetime.test.ts
• test/temp-workspace-cleanup-capability.test.ts
• test/native-worker-fd-ownership.test.ts
• test/native-init-context.test.ts
• test/value-adapter-contracts.test.ts
• test/public-api-surface.test.ts

For runtime changes, preserve the documented platform matrix and native capabilities. Stage qualification from Linux to macOS and Windows, then the remaining supported native targets. Cover Node 22/24/26, native off/auto/require, supported Bun lanes, glibc/musl, and FreeBSD's narrower native surface. Use the existing Bun native proofs. The full Bun suite is diagnostic, not an assumed passing compatibility gate. Keep cross-build success separate from execution on the target operating system.

Exercise concurrent replacement with an external process where runtime scheduling could bypass an in-process race fixture. Observe rejection, file contents, identity, descriptor ownership, cleanup, error precedence, cancellation, and event-loop delay. Successful returns alone do not prove safety or cleanup.

#scriptc feasibility gates

The frozen upstream explicitly excludes N-API addons. It also excludes computed numeric flags and BigInt filesystem stats. Its filesystem operations complete synchronously before callbacks or promise continuations. These documented limits are feasibility constraints, not newly demonstrated compiler defects.

Start with minimal unchanged-code probes, then test the actual public package. An import-only success or native-off run does not qualify the native-required path. Never replace BigInt identity with Number, retained descriptors with pathnames, or independent observations with cached values to make compilation succeed.

If a C ABI adapter is necessary, review its design before implementation. Prefer reusing the existing Rust mechanisms while keeping policy in TypeScript. Specify handle ownership, errors, cancellation, platform behavior, and ABI compatibility first. Do not create a second filesystem policy engine.

#Draft PR structure

Use dependency links only where one change actually requires another.

P1 -> P2 -> P4 forms the compiler stack. Verified fixes join that dependency chain only when necessary. Runtime work forms a separate reviewed stack. Do not couple ordinary Node fixes to compiler adoption. Coordinate OpenClaw consumer changes with their existing owners and reuse shared benchmark evidence where inputs match.

#Fork decision

Choose the owning project: tsc-ts for checker/emitter defects, scriptc for lowering/runtime defects, fs-safe for its own invariant violations. A documented unsupported feature is an architecture discussion before it is a fork justification.

Before creating a fork:

1. Preserve a minimal reproducer with expected and actual behavior.
2. Check the pinned version and current upstream for an existing fix.
3. Show that the blocker prevents an accepted fs-safe requirement.
4. Estimate the patch surface, platform tests, release ownership, and upstream synchronization cost.
5. Define an upstream contribution route and a fork retirement condition.

#Completion evidence

This plan is the first deliverable. It does not complete compiler qualification or runtime adoption. Every experiment must report pass, fail, unsupported, or not run, with its exact inputs and evidence. Keep the full objective open while required tests, verified fixes, stack preparation, or architecture decisions remain unresolved.

The final handoff must identify measured benefits, semantic differences, package/platform coverage, remaining blockers, and every draft PR dependency. Keep source validation, CI status, merge approval, release publication, and deployed behavior separate. No landing occurs without explicit maintainer approval.

#Initial Linux pilot

On Node 24.19.0 with pnpm 12.10.1, the published tsc-ts 0.0.2 Linux GNU binary passed the first comparison: 508 identical resolved input hashes, 634 identical emitted JavaScript/declaration files, and matching valid/invalid fixture results. The official-versus-official control also passed. Each pilot used two warmups and two measured samples per compiler per mode. These small pilots validate harness execution; they do not establish a speedup or full package compatibility.

Native executable SHA-256 identities:

• Official TypeScript 7.0.2: 4f2de678286401759b3fb4475bafe35b8f32b4b3a07d92642bbf37eadc9b34a4
• tsc-ts 0.0.2 Linux GNU: fb24c9cc43533dc004b9dfc9547d7066cea348b807c3239f8e31c7ba062017b6

The installed package's launcher selects lib/tsc-ts (tsc-ts.exe on Windows). Record the npm package integrity from the experiment's installation lockfile; do not infer binary provenance from its Version 7.0.2 CLI output alone.

#Linux compiler and package results

The 2026-10-17 campaign used the same frozen source and executable identities above, Linux x64 on an AWS c7a.8xlarge (32 logical CPUs), Node 24.19.0, and pnpm 12.10.1. Each comparison used two warmups and ten measured samples per compiler, in balanced ABBA/BAAB order. OS caches were uncontrolled.

Both compiler configurations preserved all 508 resolved inputs and 634 emitted JavaScript/declaration files. The complete build used the existing prepack steps with only the compiler invocation substituted in a temporary runner. Dependencies and Rust/.NET/WASM toolchains were warm; dist was recreated on each invocation. All 636 complete-build artifacts matched across every run. The median build saving was 22.1%, approximately 293 ms.

GNU time measured median build user CPU of 2.890/1.955 seconds, system CPU of 0.715/0.475 seconds, and maximum RSS of 310,372/261,192 KiB (official/candidate). Maximum RSS is a process high-water mark, not summed concurrent memory. Compiler-only CPU and RSS were not measured.

Candidate output also passed the installed-package checker without a rebuild: public imports and API manifest, guest Python/rename protocol, directory receipt runtime behavior, Root.walk declarations, and the three consumer fixtures under both compilers. The complete dist inventory was unchanged after packing.

This clears the provisional Linux performance gate. It does not qualify cold builds, macOS/Windows compiler binaries, other runtimes, or every native mode. Identical emitted JavaScript provides no evidence of faster filesystem I/O. Keep compiler adoption separate from these qualification tools.

#Platform and clean-output campaign

compiler-qualification.yml runs the published tsc-ts 0.0.2 native package on Linux, macOS, and Windows with Node 24.19.0. It records npm lockfile integrity, native executable hashes, the exact checkout commit, and toolchain versions. Windows uses the existing x64 CI runner; upstream 0.0.2 has no Windows ARM64 package. The npm launcher gets a separate smoke check using its default libraries. Parity and package consumers continue to use the official TypeScript libraries.

The campaign runs the compiler parity harness in default and single-threaded modes, with two warmups and ten measured samples per arm. Complete builds use the existing prepack owner with an explicit native compiler for both arms. Ordinary builds retain the official compiler; this option is experimental qualification tooling, not an adoption decision. Strict toolchain mode requires the .NET helper build and cannot reuse the staged executable as a fallback.

Warm complete builds retain Rust/.NET outputs and recreate dist. Clean-output builds use a fresh Cargo target and remove only the helper's bin and obj directories before each invocation. Dependencies and their download stores stay installed; OS and dependency caches are uncontrolled. These are clean build outputs, not cold-machine or cold-page-cache measurements. Each arm has two warmups, ten measured warm builds, and six measured clean-output builds, in balanced ABBA/BAAB order. Timings cover the complete build, excluding hashing, output cleanup and installed-package checks. They measure wall time only.

Every build must match all JavaScript, declaration, helper, and WASM bytes on that runner. Cross-OS binary equality is not required. Source, build scripts, lockfiles, generated helper input, compiler binaries, and libraries are checked for mutation. Both arms in each cache mode get installed-package proof without a rebuild, followed by another complete dist hash comparison.

The runner accepts only its disposable Actions checkout. The isolated job's 45-minute deadline owns cancellation of the full Cargo/.NET process tree; it does not use a parent-only build timeout and then continue measuring. It uploads at most three redacted JSON reports per platform, each capped at 4 MiB, for 14 days. Generated binaries and compiler installations stay temporary. A failed campaign cannot establish a performance claim, even if it has samples.

OpenClaw is a separate qualification target. Its declaration build uses typescript/unstable API transport and program.emitToString; a faster CLI typecheck does not establish compatibility with that build owner. Reuse the existing compiler lane's exact-source evidence before proposing an API/emission adapter or a paired build:strict-smoke comparison.

#Separate scriptc runtime probe

Published scriptc 0.3.5 was installed on the same Linux host. Its native control program built and ran after the host's missing Clang was installed. The executable SHA-256 was b7087211a304c049de53fcaac40564e9a6e9adac6b4055c895ab340e82d47a84. Its release tag is a source reference, not an attestation of the npm binary.

All three filesystem probes ran successfully in Node. With scriptc coverage configured to fail on blockers:

• statSync(path, { bigint: true }) failed preflight with SC0001: the

supported declaration accepts only one argument.

• Passing computed numeric open flags failed lowering with SC2020.
• An installed-package consumer using root(path).readBytes(...) failed

preflight with missing BigIntStats and NodeJS.Platform declarations (SC0001), plus the package's computed/escaping createRequire use (SC2013).

These results agree with the release's documented descriptor/identity limits and native-addon boundary. They block runtime adoption; they do not establish a newly broken supported contract or justify a fork. No fs-safe safety policy was weakened for these probes. Further native-port design requires an exact-identity and addon/FFI solution before broader runtime tests can become meaningful.