Directory durability

@openclaw/fs-safe/durability provides the directory side of crash-safe file publication. Flushing a file does not guarantee that its containing directory entry reached storage; callers that promise durable create, link, rename, or unlink operations must also synchronize the affected directory.

import {
  ensureDurableDirectory,
  pinDirectory,
} from "@openclaw/fs-safe/durability";

const repository = await ensureDurableDirectory({
  directoryPath: "/srv/backups/sqlite",
  mode: 0o700,
});

const pinned = await pinDirectory(repository, { label: "backup repository" });
try {
  await publishSnapshot();
  const outcome = await pinned.sync();
  if (outcome.status === "unsupported") {
    // Decide at the product boundary whether this platform can weaken the promise.
  }
} finally {
  await pinned.close();
}

#Outcomes and failure semantics

syncDirectory() and PinnedDirectory.sync() return:

type DirectorySyncOutcome =
  | { status: "synced" }
  | { status: "unsupported"; code?: string };

POSIX synchronization failures propagate. Windows directory handles do not portably support FlushFileBuffers; the known unsupported error family is reported as unsupported after the pathname and pinned identity are checked again. Directory-open access failures and other Windows I/O failures still propagate.

syncDirectoryBestEffort() and syncDirectoryBestEffortSync() intentionally discard both unsupported outcomes and failures. Use them only when the primary write remains useful without a crash-durability promise.

On Windows, pathname inputs and supplied directory receipts reject NTFS alternate-stream and directory-index namespace spellings before opening, creating, hashing, or publishing anything. This applies to directory durability, publishFileExclusive(), and the pathname overloads of sha256File() and sha256FileSync(); the already-open FileHandle and borrowed numeric file-descriptor overloads are unchanged. Ordinary colon-bearing POSIX paths remain valid.

#Pinned directories

pinDirectory() rejects final symlinks and non-directories. On POSIX it opens with O_DIRECTORY, O_NOFOLLOW, and O_NONBLOCK, then compares the open descriptor, pathname identity, and canonical path. assertCurrent() repeats those checks. This prevents a pathname replacement from turning a later sync into proof for a different directory.

Pathname and descriptor checks compare exact bigint device and inode values. Each inspection allows one retry for unknown Windows identity components, retaining known components and rejecting definite mismatches immediately. Persistent unknown identity fails closed with path-mismatch, including on an otherwise usable directory. A failed preflight identity check never syncs the descriptor; a replacement discovered after sync still rejects the operation.

DirectoryReceipt.identity remains a numeric Node Stats object for metadata compatibility. Its exactIdentity companion contains immutable bigint dev and ino, projected from the same observation. Library-created receipts and their identity objects retain a private exact snapshot; mutating their public fields cannot change the directory authorized by that snapshot. Each returned receipt owns a mutable numeric metadata copy. Later admissions retain the original metadata snapshot even if advisory fields such as mode or timestamps were edited; changed paths or identity components reject. Pass the receipt or its original identity object through to later operations to retain this evidence. When copying a receipt, preserve exactIdentity; pass it to expectedExistingIdentity. A copied numeric identity without its exact companion is accepted only when both components are safe integers and, on Windows, nonzero. Rounded or unknown caller identities fail with path-mismatch rather than authorizing a different directory.

Caller-supplied receipts may use DirectoryReceipt<BigIntStats> with the result of lstat(path, { bigint: true }). pinDirectory(), syncDirectory(), syncDirectorySync(), publishFileExclusive()'s parentReceipt, and stageFileInDirectory() accept both numeric and bigint receipt inputs. DirectoryReceipt without a type argument and all returned durability receipts still expose numeric Stats, including working type predicates and Date properties. Bigint metadata is projected from the supplied observation, retaining fractional timestamps and the private exact device/inode identity.

import { lstatSync, realpathSync, type BigIntStats } from "node:fs";
import { syncDirectorySync, type DirectoryReceipt } from "@openclaw/fs-safe/durability";

const directoryPath = "/srv/backups/sqlite";
const receipt: DirectoryReceipt<BigIntStats> = {
  path: directoryPath,
  realPath: realpathSync(directoryPath),
  identity: lstatSync(directoryPath, { bigint: true }),
};
// Keep this receipt across the application's publication operation.
const outcome = syncDirectorySync(receipt);

Call close() in finally. Closing is idempotent; using a closed pin fails.

These checks intentionally reject a moved or replaced pathname. For one file's abort cleanup through its original directory after a move, use the separate retained-directory staging lifecycle. Its cleanup authority does not weaken pinDirectory().assertCurrent() or .sync(), and namespace cleanup is not proof of crash durability.

#Durable directory creation

ensureDurableDirectory() finds and pins the nearest existing ancestor, creates the requested path, opens every new directory segment, and synchronizes each new parent-to-child edge from the leaf upward. It returns the final directory receipt plus the aggregate parent-sync outcome.

By default it uses fs-safe's guarded one-segment-at-a-time absolute-directory creator. Advanced callers can pass create when directory creation needs platform-specific ACLs or another product-owned policy. The callback owns the safety of its mutations and must create exactly directoryPath; fs-safe validates and pins every resulting segment before any synchronization is accepted.

expectedExistingIdentity binds an existing target to an identity observed by the caller before a separate permission or policy check. A missing or replaced target fails with FsSafeError("path-mismatch"). Use bigint dev and ino from lstat(path, { bigint: true }) or readDirectoryIdentity() for caller-owned observations that may exceed the numeric safe-integer range. An original library receipt's identity object also retains its private exact identity for this option.

#Exclusive file publication

publishFileExclusive() materializes one file without clobbering an existing target. It pins the source with nonblocking O_NOFOLLOW, optionally verifies expectedSourceIdentity, tries a hardlink first, then synchronizes the target parent directory.

Pass expectedSourceIdentity from fs.stat(path, { bigint: true }) (or fs.lstat) for exact Windows file IDs. Unsafe Number components cannot prove identity and reject with FsSafeError("path-mismatch") before publication; safe integer Numbers remain supported. Internal source fences use bigint stats.

Every successful publication returns exactIdentity: Readonly<{ dev: bigint; ino: bigint }> alongside the unchanged identity: Stats metadata view. Both come from one final bigint fstat of the retained descriptor. Pass result.exactIdentity back as expectedSourceIdentity when publishing that file again. Passing the whole publication result is also supported: its explicit exactIdentity takes precedence. sameFileIdentity() and copyRootFileSync().expectedSourceIdentity accept that same receipt shape.

On Windows, the Number view may round an NTFS file ID; it must not be used for identity checks. Do not re-stat a pathname to reconstruct an earlier publication receipt: that can observe a replacement. After a failed publication, PublishFileExclusiveFailureDetails.exactTargetIdentity identifies the created target when known; targetIdentity remains the legacy numeric view. Retain the exact failure field when inspecting a preserved target. Unknown or mismatching identities still fail closed.

Trusted relative source and target paths are resolved to absolute paths before authority checks. On Windows, ordinary drive-relative operands are anchored at entry before namespace-alias admission. A caller-supplied parentReceipt must still name the resolved target parent and is not relaxed by this compatibility rule.

For example, a backup archive is complete before publication. If directory sync fails, keeping that complete file is more useful than conditionally deleting it by pathname:

import { FsSafeError } from "@openclaw/fs-safe/errors";
import { publishFileExclusive } from "@openclaw/fs-safe/durability";

try {
  const result = await publishFileExclusive({
    sourcePath: stagedArchive,
    targetPath: finalArchive,
    strategy: "link-or-copy",
    onSyncFailure: "preserve",
    parentReceipt: backupDirectory,
  });
  recordDurableBackup(result.exactIdentity, result.directorySync);
} catch (error) {
  if (
    error instanceof FsSafeError &&
    error.details?.phase === "directory-sync" &&
    error.details.cleanup === "preserved"
  ) {
    recordCompleteButPossiblyNonDurableBackup(finalArchive, error.details);
  } else {
    throw error;
  }
}

#Strategies

rename-noreplace normally uses atomic native rename. On Linux in native auto mode, a definitively unsupported RENAME_NOREPLACE result (EINVAL, ENOSYS, or unsupported-operation errno) permits the existing native file-move fallback. It requires a regular source with one link, retains both parents, creates the target with linkat without following symlinks, verifies both names against the staged identity with exactly two links, then unlinks the source and verifies the target has one link. Fallback collisions report already-exists without clobbering; the atomic rename path retains its existing EEXIST error. Success still returns method: "rename-noreplace", with the additive fallback: "link-unlink" field identifying this weaker mechanism. Atomic rename and the other strategies omit fallback.

Link/unlink is not crash-atomic. A crash between those syscalls can leave both names on the same complete inode. Callers must recognize that interrupted pair as incomplete evidence or recover it under their own identity/ownership rules before accepting a sealed artifact. The source parent is synchronized after unlink, and the target parent is synchronized through the existing pinned directory/parentReceipt path before success. These directory barriers do not flush file contents: callers must finish and synchronize staged contents first. Identity checks and syscalls remain separate, not source-name compare-and-swap; detected substitutions refuse completion and preserve evidence.

Native require remains fail-closed with helper-unavailable and details.capability === "rename-noreplace". Missing helpers, native off, and non-Linux platforms gain no fallback. If linkat returns EPERM, EOPNOTSUPP, EMLINK, EXDEV, or ENOSYS, publication also fails closed, naming both rename-noreplace and details.fallbackCapability: "linkat". Permission or link count limits need not mean the entire filesystem lacks hard-link support. link-required and link-or-copy do not use renameat2 and are unchanged.

"link-required" propagates an unsupported hardlink failure. "link-or-copy" falls back only for EACCES, EPERM, EXDEV, ENOTSUP, EOPNOTSUPP, or ENOSYS; isHardlinkFallbackError() exposes that exact classifier, including Android/Termux hardlink permission refusals. A genuine permission denial still fails the fallback's exclusive target creation; EEXIST never permits fallback. The fallback copies from the pinned source into a wx target, fsyncs it, and fences source and target identity and content before reporting success. parentReceipt, when supplied, must name the target's direct parent.

With a native binding, the copy fallback first attempts a copy-on-write clone (fclonefileat on macOS, FICLONE on Linux), then Linux copy_file_range, and finally the existing JavaScript byte loop. Every route creates the target exclusively, applies mode 0o600 through its owned descriptor independently of the process umask, and goes through the same post-copy identity and SHA-256 fencing. Hashing uses an async native task when available, so large verification reads do not occupy the JavaScript event loop.

On a clone-capable filesystem, publication of a large file becomes mostly a metadata operation: data blocks are shared copy-on-write until either file is modified. Clone support is filesystem- and mount-dependent, so callers must not infer durability or physical independence from timing; an unsupported clone or copy_file_range transparently continues down the fallback chain.

#Recoverable atomic-replace fallback

replaceFileAtomic() normally publishes a sibling temp with an atomic rename; file and directory synchronization are opt-in. Its permission-error copy fallback remains non-atomic even with opt-in copyFallbackRestore: "restore-original". Recovery requires an explicit maxRestoreBytes budget and inspection of the details.cleanup receipt. See Atomic writes for restoration limits, identity/authority refusals, and retained-inode semantics.

#Streaming SHA-256

sha256File() hashes either a pathname string or an already-open Node FileHandle. A backup verifier can pin the file itself, compare its size, and keep ownership of the handle:

import { open } from "node:fs/promises";
import { sha256File } from "@openclaw/fs-safe/durability";

const snapshot = await open(stagedArchive, "r");
try {
  const before = await snapshot.stat();
  const hash = await sha256File(snapshot, { maxBytes: before.size });
  if (hash.bytes !== before.size || hash.digest !== manifest.sha256) {
    throw new Error("staged backup does not match its manifest");
  }
} finally {
  await snapshot.close();
}

The result is { bytes, digest }, where digest is lowercase hexadecimal. The optional Sha256FileOptions argument supports maxBytes and signal. maxBytes accepts a non-negative safe integer (including zero), or Infinity for no limit, which is also the default. Oversized files reject with FsSafeError("too-large"); reads stay bounded to at most maxBytes + 1 bytes, even if the file grows after its initial size check. Hashing never silently truncates to the limit.

Pass signal to cancel. A pre-aborted signal rejects before file I/O or native loading. In-flight cancellation is cooperative between reads and rejects with the signal's original reason only after the pending read or native task stops. Callers may close their handle after awaiting rejection; do not close it while the operation is pending. A signal can be shared by successive or concurrent hashes. Neither mode provides a snapshot of concurrently modified contents; callers requiring stable content must also fence identity and metadata.

The handle overload never closes the caller's descriptor and uses positioned reads, so it does not alter the descriptor's current offset. The path overload rejects symbolic links and non-regular files, compares lossless bigint identities from the pre-open pathname inspection to the opened descriptor and from that descriptor to the current pathname, and closes its own handle. All identity checks complete before any JavaScript or native hashing. Each inspection allows one bounded retry for unknown Windows identity components, retaining known components and rejecting known differences immediately. Persistent unknown identity fails closed with path-mismatch, even for benign files: this trades availability for verifiable identity. Retries inspect the same descriptor or pathname without reopening the file and repeat the symlink and file-type checks. POSIX opens are nonblocking, so a raced FIFO or device is rejected after descriptor inspection rather than waiting for a writer.

A pathname hash reports failure to close its owned descriptor after successful hashing. If hashing, admission, or cancellation already failed, that original failure remains primary even when close also fails. This also applies to sha256FileSync(); borrowed handles and descriptors remain caller-owned.

When the optional binding is active, hashing runs as an async native task and does not occupy the JavaScript event loop with digest updates. With native mode off, or in auto when no binding loads, the fallback performs asynchronous positioned reads in chunks of up to 256 KiB but updates Node's Hash on the JavaScript thread. Both paths stream bounded buffers rather than loading the file into memory. The fallback sizes its scratch buffer to small files and grows it if a stale size hint is exceeded, while still probing for actual EOF and byte-limit overflow. Native mode require keeps its usual fail-closed loader semantics. On Windows, pathname hashing can read through its privately owned handle without reopening it. Hashing a caller-owned handle still uses an independent reader to preserve the caller's position.

#Synchronous hashing

sha256FileSync() accepts a pathname or a borrowed numeric file descriptor and returns the same { bytes, digest } result. It shares Sha256FileOptions, including the default unlimited byte budget and too-large errors for growth beyond maxBytes. It always reads from offset zero with bounded positional readSync calls and leaves a borrowed descriptor open at its original position. Path inputs use the same regular-file, final-symlink, nonblocking-open, and exact bigint admission checks described above, then close their owned descriptor. These checks do not provide ancestor confinement or a snapshot of concurrent edits.

import { closeSync, openSync } from "node:fs";
import { sha256FileSync } from "@openclaw/fs-safe/durability";

const fd = openSync(stagedArchive, "r");
try {
  const hash = sha256FileSync(fd, { maxBytes: manifest.sizeBytes });
  if (hash.bytes !== manifest.sizeBytes || hash.digest !== manifest.sha256) {
    throw new Error("staged backup does not match its manifest");
  }
} finally {
  closeSync(fd);
}

The synchronous API uses Node's crypto implementation in every native mode, including require; it never loads a native binding. It blocks the calling thread until hashing finishes or throws. A pre-aborted signal fails before I/O, and synchronous signal changes are checked between operations with the original reason preserved. Timers and other JavaScript callbacks cannot run while the hash is executing; use sha256File() when responsive cancellation is needed.

#Publication failure receipts

If publishFileExclusive() fails after creating the target, it throws an FsSafeError with a details receipt:

type PublishFileExclusiveFailureDetails = {
  phase:
    | "hardlink-create" | "hardlink-verify"
    | "copy-create" | "copy-verify"
    | "rename-create" | "rename-verify"
    | "directory-sync";
  targetCreated: boolean;
  targetIdentity?: { dev: number | bigint; ino: number | bigint };
  cleanup: "removed" | "preserved" | "unknown";
  directorySync?: { status: "failed"; code?: string };
  fallback?: "link-unlink";
  publication?: "published";
  sourceRemoval?: "not-attempted" | "still-linked" | "unverified" | "removed";
};

"removed" means the path still matched the identity created by this call and was unlinked (or was already absent). "preserved" means it was deliberately retained—for example after a successful no-replace rename—or the pathname had been replaced and therefore was not safe to remove. "unknown" means cleanup could not verify or remove the created identity. Callers that run a second application-level guard, such as SQLite snapshot validation, should branch on this receipt instead of inferring ownership from path existence. The original failure remains available as cause. Failures before target creation retain their existing error shape and do not claim a cleanup result.

After a fallback link succeeds, failure always preserves the target, irrespective of onSyncFailure. The receipt reports fallback: "link-unlink", publication: "published", and the source-removal state. still-linked means unlink failed and both names were reverified as the exact two-link pair; not-attempted means pair verification failed before unlink; unverified means unlink failed and the pair could no longer be verified; removed means unlink completed but a later verification or sync failed. Publication describes the completed link operation, not continuing authority over the target pathname. Unverified targets do not receive a claimed targetIdentity.

Source and target identities are checked again after successful or unsupported directory synchronization, while their descriptors remain owned. A late verification failure retains its strategy's verification phase and is not a directory-sync failure. Completed copied targets stay pinned during conditional cleanup; substituted entries remain untouched. Returned numeric metadata comes from the retained target descriptor and grants no continuing pathname authority.

#Directory-sync failure policy

onSyncFailure applies only after target creation and content/identity fencing have succeeded but synchronizing the containing directory throws:

type PublishFileExclusiveSyncFailurePolicy = "rollback" | "preserve";

A returned { status: "unsupported", code? } is an explicit successful publication outcome, not a thrown sync failure, so this option does not rewrite or clean up that target.

• rollback is the default. fs-safe removes the target only if its current

identity still matches the file created by this call. A replacement is never removed. The error reports cleanup: "removed", "preserved", or "unknown" and directorySync: { status: "failed", code? }.

• preserve never attempts that unlink. The error reports

targetCreated: true, cleanup: "preserved", the created identity, and the failed directory-sync outcome. The file is complete and fenced, but its directory entry is not proven crash-durable.

Choose rollback when the pathname must mean “durably committed” and a failed commit should disappear from the live process view. Choose preserve when the payload itself remains valuable—backup archives are the common case—and the caller can record, retry, or independently validate durability. Neither choice can make a failed directory sync succeed: rollback deletion is also not proven durable, and a preserved name may disappear after a crash. Always use the typed receipt rather than inferring ownership from exists().

rename-noreplace always preserves its target after a successful rename or fallback link, because removing it would discard the source's only remaining name; its typed failure receipt makes that explicit regardless of onSyncFailure.

"rename-noreplace" requires the native helper and moves the source to the target without replacement, with the Linux auto fallback limits described above. A collision is reported as EEXIST for atomic rename or already-exists for its Linux fallback, both files remain unchanged, and a successful call returns method: "rename-noreplace" after synchronizing the source and target parent directories. Unlike the link/copy strategies, success consumes sourcePath.

#Scope

These primitives establish path identity and filesystem synchronization. One publishFileExclusive() call is one no-clobber file materialization, not a retention policy, multi-file transaction, or application commit protocol. They do not decide application commit protocols, marker formats, permission policy, or whether an unsupported platform is acceptable. Keep those decisions at the owning product boundary.

#See also

• Migrating to 0.5 — choosing a publication policy during upgrade.
• Native architecture — clone/copy/hash mechanisms and fallback guarantees.
• Errors — typed operational failure handling.

#Existing Windows file retirement

retainFileInDirectory describes identity-bound native disposition and resource settlement separately from persistence. Its result is always persistence: "not-proven"; neither accepted disposition nor observed namespace absence is a directory/volume barrier or an application commit.