fix(run-ops): DNS-safe, sortable base32hex run id (replace base62 KSUID)

[d-cs]

Problem

The run-ops split mints NEW-store run ids as 27-char base62 KSUIDs. The supervisor writes the run id into the Kubernetes pod name (runner-<id>), and pod names must be DNS-1123 labels (lowercase [a-z0-9-]) — so uppercase base62 ids make k8s reject the pod (422) and those runs never launch (they loop in PENDING_EXECUTING until the heartbeat-stall handler nacks them, forever). .toLowerCase() can't fix it: base62 has both A(10) and a(36) as distinct symbols, so folding collides distinct ids and destroys sort order.

Fix: change the encoding, not the structure

Mint a 26-char lowercase base32hex run id:

run_<24-char base32hex core><region char><version char>
      [ 6-byte ms timestamp ][ 9 CSPRNG bytes ]

• base32hex (RFC 4648 §7, alphabet 0-9a-v): lowercase, order-preserving, DNS-safe; 15 bytes → exactly 24 chars, no padding. Hand-rolled encode/decode (no new dependency).
• 48-bit ms timestamp in the leading bytes → plain string sort == creation order at millisecond resolution.
• 72 bits CSPRNG entropy; PK unique constraint is the backstop (no retry loop).
• region / version are raw positional chars (read via one charAt before decoding/routing), version = "1".

DNS-safe from birth and hyphen-free, so firekeeper is unchanged — runner-<id>-attempt-N → strip runner-, cut at first hyphen still recovers the exact id incl. region+version.

Residency discriminator: length → version char

classifyKind/classifyResidency (runOpsResidency.ts) previously distinguished NEW vs LEGACY by id length. That gets ambiguous with a third format. It now discriminates on the version char at a fixed position (isRunOpsIdBody: 26 chars, [25] === "1", base32hex alphabet) → NEW; everything else → LEGACY. Total, never throws. The Residency (NEW/LEGACY) contract the routing store consumes is unchanged; the "ksuid" ResidencyKind label is retained only because it's the persisted runOpsMintKsuid feature-flag value.

Scope / verification

• Generator + discriminator in @trigger.dev/core isomorphic; mint path + all id-shape call sites swept (~40 webapp files); changeset added (@trigger.dev/core patch).
• Core unit tests (encode/decode round-trip + property, generator shape, ms sort-order incl. intra-second, parse partitioned-vs-legacy, firekeeper round-trip): 24 pass. @trigger.dev/core builds; webapp typechecks; format/lint clean.

Open decisions (flagged, not silently chosen)

1. Backward-compat: existing 27-char base62 KSUID runs now classify LEGACY. On test cloud these are the broken/looping runs that never completed, so this is acceptable — but worth a conscious call before prod. No transitional length-recognition added (keeps the discriminator clean).
2. Storage collation: the sort guarantee is byte-order — if the run-ops id column is TEXT with default locale collation it's silently not honored. Confirm whether COLLATE "C" / BYTEA is needed on the run-ops schema.
3. Region sourcing wiring — see regionCharForRegion / REGION_CODES.

---

⚠️ Required migration — deploy in lockstep

This PR renames a persisted feature-flag key/value and an env var. These are not changed by the code alone and must be migrated when this deploys, or affected orgs silently fall back to cuid minting (no crash — defaultValue: "cuid"):

1. Env var (terraform): RUN_OPS_MINT_KSUID_ENABLED → RUN_OPS_MINT_ENABLED (carry the value over).
2. DB organization.featureFlags: migrate both the key and value together:
   • key runOpsMintKsuid → runOpsMintKind
   • value "ksuid" → "runOpsId"

Until an org's flag row is migrated, its runOpsMintKind lookup misses and it mints cuid (legacy) — so no NEW-store ids for that org until the data lands.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 305ed4a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Walkthrough

This PR replaces KSUID-based run identifiers with run-ops v1 identifiers across core generation/parsing, residency classification, webapp minting/routing, run-engine behavior, run-store routing, and related tests. The core package adds run-ops ID helpers and updates residency kinds and classifiers. Webapp services, routes, and tests now mint, classify, and route run-ops IDs, including region-aware child IDs and the new runOpsId mint kind. Run-engine and run-store logic, fixtures, comments, and tests are updated to use the new 26-character shape and version-character classification.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main run-ops ID format change and the KSUID replacement.
Description check	✅ Passed	The description is thorough and covers the problem, fix, verification, and migration, though it does not follow the template headings exactly.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/run-ops-ulid-run-id

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

apps/webapp/test/apiRunResultPresenter.readthrough.test.ts (1)

24-28: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

newFriendlyId() needs the 26-char v1 body

newFriendlyId() still returns a 27-char body, so the NEW-path cases are seeding an id that the residency rule treats as LEGACY. Update it to the 26-char v1 shape to match the classification logic.

🧹 Nitpick comments (2)

internal-packages/run-engine/src/engine/tests/lifecycleRouter.test.ts (1)

459-466: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Sanitizing regex could collapse distinct suffixes into the same id.

newRunId replaces any char outside [0-9a-v] with "0". Two suffixes differing only in letters outside that range (e.g. w-z, uppercase) would sanitize to identical strings, silently producing duplicate ids and masking test bugs. Current call sites (e.g. "new_run_e"/"new_run_g") don't trigger this, but it's a latent footgun for future test authors.

apps/webapp/test/engine/triggerTask.test.ts (1)

2396-2403: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Redundant dynamic import() — use the already-imported generateRunOpsId.

generateRunOpsId is statically imported in this file (line 33), so the dynamic import("@trigger.dev/core/v3/isomorphic") here is unnecessary.

♻️ Proposed fix

-        (await import("`@trigger.dev/core/v3/isomorphic`")).generateRunOpsId()
+        generateRunOpsId()

Based on coding guidelines: "Prefer static imports over dynamic import(); only use dynamic imports when resolving circular dependencies, enabling real code splitting, or conditionally loading a module at runtime."

Source: Coding guidelines

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 4c495098-38e7-4d88-9358-5915b732f109

📥 Commits

Reviewing files that changed from the base of the PR and between 119189f and 48a6686.

📒 Files selected for processing (66)

• .changeset/olive-bees-repeat.md
• apps/webapp/app/runEngine/concerns/idempotencyResidency.server.test.ts
• apps/webapp/app/runEngine/services/triggerFailedTask.server.ts
• apps/webapp/app/runEngine/services/triggerTask.server.ts
• apps/webapp/app/utils/friendlyId.test.ts
• apps/webapp/app/utils/friendlyId.ts
• apps/webapp/app/v3/runOpsMigration/crossSeamGuard.server.ts
• apps/webapp/app/v3/runOpsMigration/mintBatchFriendlyId.server.test.ts
• apps/webapp/app/v3/runOpsMigration/mintBatchFriendlyId.server.ts
• apps/webapp/app/v3/runOpsMigration/readThrough.server.test.ts
• apps/webapp/app/v3/runOpsMigration/resolveInheritedMintKind.server.test.ts
• apps/webapp/app/v3/runStore.server.test.ts
• apps/webapp/app/v3/services/batchTriggerV3.server.ts
• apps/webapp/app/v3/services/bulk/BulkActionV2.batchReadThrough.server.test.ts
• apps/webapp/app/v3/services/executeTasksWaitingForDeploy.ts
• apps/webapp/test/SpanPresenter.readthrough.test.ts
• apps/webapp/test/api.v1.waitpoints.tokens.test.ts
• apps/webapp/test/apiBatchResultsPresenter.readroute.test.ts
• apps/webapp/test/apiBatchResultsPresenter.readthrough.test.ts
• apps/webapp/test/apiRetrieveRunPresenter.readroute.test.ts
• apps/webapp/test/apiRunResultPresenter.readthrough.test.ts
• apps/webapp/test/apiWaitpointPresenter.readthrough.test.ts
• apps/webapp/test/batchTriggerV3ResidencyInheritance.test.ts
• apps/webapp/test/batchTriggerV3StoreRouting.test.ts
• apps/webapp/test/bulkActionV2ReadRouting.test.ts
• apps/webapp/test/cancelDevSessionRunsStoreRouting.test.ts
• apps/webapp/test/crossSeamGuard.proof.test.ts
• apps/webapp/test/engine/triggerFailedTask.test.ts
• apps/webapp/test/engine/triggerTask.test.ts
• apps/webapp/test/idempotencyDedupResidency.test.ts
• apps/webapp/test/idempotencyKeyConcernLegacyAuthority.test.ts
• apps/webapp/test/performTaskRunAlertsStoreRouting.test.ts
• apps/webapp/test/realtime/runReaderReadThrough.test.ts
• apps/webapp/test/resetIdempotencyKeyLegacyAuthority.test.ts
• apps/webapp/test/resolveWaitpointThroughReadThrough.readthrough.test.ts
• apps/webapp/test/runOpsCrossSeamGuard.test.ts
• apps/webapp/test/runOpsMintCutover.test.ts
• apps/webapp/test/runPresenterReadRoute.test.ts
• apps/webapp/test/runsRepository.readthrough.test.ts
• apps/webapp/test/sessions.readthrough.test.ts
• apps/webapp/test/updateMetadataStoreRoutingHetero.test.ts
• internal-packages/run-engine/src/engine/systems/executionSnapshotSystem.test.ts
• internal-packages/run-engine/src/engine/systems/waitpointSystem.test.ts
• internal-packages/run-engine/src/engine/tests/blockEdgeResidency.test.ts
• internal-packages/run-engine/src/engine/tests/completeWaitpointReadResidency.test.ts
• internal-packages/run-engine/src/engine/tests/datetimeWaitpointColocation.test.ts
• internal-packages/run-engine/src/engine/tests/lifecycleRouter.test.ts
• internal-packages/run-engine/src/engine/tests/triggerCreateRouting.test.ts
• internal-packages/run-store/src/PostgresRunStore.batchProbeReadClient.test.ts
• internal-packages/run-store/src/PostgresRunStore.controlPlaneAlertFk.test.ts
• internal-packages/run-store/src/PostgresRunStore.dedicatedRepro.test.ts
• internal-packages/run-store/src/PostgresRunStore.writeAtomicity.test.ts
• internal-packages/run-store/src/batchCompletionResidency.test.ts
• internal-packages/run-store/src/runOpsStore.flipWindowDuplicate.test.ts
• internal-packages/run-store/src/runOpsStore.forWaitpointCompletion.test.ts
• internal-packages/run-store/src/runOpsStore.idempotencyDedup.test.ts
• internal-packages/run-store/src/runOpsStore.mixedResidency.test.ts
• internal-packages/run-store/src/runOpsStore.readAfterWrite.test.ts
• internal-packages/run-store/src/runOpsStore.snapshots.test.ts
• internal-packages/run-store/src/runOpsStore.test.ts
• internal-packages/run-store/src/runOpsStore.ts
• internal-packages/run-store/src/runOpsStore.waitpoints.test.ts
• packages/core/src/v3/isomorphic/friendlyId.test.ts
• packages/core/src/v3/isomorphic/friendlyId.ts
• packages/core/src/v3/isomorphic/runOpsResidency.test.ts
• packages/core/src/v3/isomorphic/runOpsResidency.ts

💤 Files with no reviewable changes (1)

• apps/webapp/app/v3/runOpsMigration/crossSeamGuard.server.ts

[d-cs]

Addressed the CodeRabbit review (folded into this branch, latest commit):

• newFriendlyId() returned a 27-char body (NEW-path cases silently seeding LEGACY-classified ids) — fixed to a valid 26-char v1 body ("run_" + generateRunOpsId()). Audited the other copies: they already used a correct 26-char body.
• newRunId (lifecycleRouter.test) lossy →"0" sanitization that could collapse distinct suffixes — rewritten to map into the base32hex alphabet by code point (no collisions).
• triggerTask.test dynamic import() — replaced with the statically-imported generateRunOpsId().

Also in this branch: the stale ksuid identifiers were renamed to runOpsId (persisted wire values — the runOpsMintKsuid flag key, RUN_OPS_MINT_KSUID_ENABLED env, and the "ksuid"/"cuid"enum values — deliberately left unchanged), a comment-clarity pass, and latestmain` merged in (so it includes #4153).

[pkg-pr-new]

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@9cfda5a

trigger.dev

npm i https://pkg.pr.new/trigger.dev@9cfda5a

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@9cfda5a

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@9cfda5a

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@9cfda5a

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@9cfda5a

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@9cfda5a

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@9cfda5a

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@9cfda5a

commit: 9cfda5a

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

scripts/retry-prisma-generate.mjs (2)

24-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Surface result.error on spawn failure.

If spawnSync itself fails to launch prisma (e.g., not on PATH), result.stdout/result.stderr are empty, the transient-error regex never matches, and the script exits with an unhelpful generic status. Printing result.error would make that failure mode debuggable.

♻️ Proposed diff

   process.stdout.write(result.stdout ?? "");
   process.stderr.write(result.stderr ?? "");
+
+  if (result.error) {
+    console.error(`prisma generate failed to spawn: ${result.error.message}`);
+  }

---

25-28: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Consider scoping shell: true to Windows only.

The header comment explains this wrapper exists to work around a Windows-only EPERM race. Unconditionally enabling shell: true on all platforms forwards passthroughArgs through shell interpretation unnecessarily on POSIX systems.

♻️ Proposed diff

   const result = spawnSync("prisma", ["generate", ...passthroughArgs], {
-    shell: true,
+    shell: process.platform === "win32",
     encoding: "utf8",
   });

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 19b18649-cad8-47f7-92f3-9b336fc31984

📥 Commits

Reviewing files that changed from the base of the PR and between 40fca04 and ba796ec.

📒 Files selected for processing (26)

• .changeset/olive-bees-repeat.md
• .github/workflows/e2e.yml
• apps/webapp/app/env.server.ts
• apps/webapp/app/runEngine/concerns/idempotencyResidency.server.test.ts
• apps/webapp/app/runEngine/concerns/idempotencyResidency.server.ts
• apps/webapp/app/runEngine/services/triggerFailedTask.server.ts
• apps/webapp/app/runEngine/services/triggerTask.server.ts
• apps/webapp/app/v3/featureFlags.ts
• apps/webapp/app/v3/runOpsMigration/mintBatchFriendlyId.server.test.ts
• apps/webapp/app/v3/runOpsMigration/mintBatchFriendlyId.server.ts
• apps/webapp/app/v3/runOpsMigration/resolveInheritedMintKind.server.test.ts
• apps/webapp/app/v3/runOpsMigration/resolveInheritedMintKind.server.ts
• apps/webapp/app/v3/runOpsMigration/runOpsMintKind.flipLatency.test.ts
• apps/webapp/app/v3/runOpsMigration/runOpsMintKind.server.test.ts
• apps/webapp/app/v3/runOpsMigration/runOpsMintKind.server.ts
• apps/webapp/app/v3/services/batchTriggerV3.server.ts
• apps/webapp/test/batchTriggerV3ResidencyInheritance.test.ts
• apps/webapp/test/engine/triggerFailedTask.test.ts
• apps/webapp/test/engine/triggerTask.test.ts
• apps/webapp/test/runOpsMintCutover.test.ts
• internal-packages/database/package.json
• internal-packages/run-ops-database/package.json
• internal-packages/run-store/src/runOpsStore.routedReadPrimary.test.ts
• packages/core/src/v3/isomorphic/runOpsResidency.test.ts
• packages/core/src/v3/isomorphic/runOpsResidency.ts
• scripts/retry-prisma-generate.mjs

✅ Files skipped from review due to trivial changes (4)

• .github/workflows/e2e.yml
• .changeset/olive-bees-repeat.md
• apps/webapp/app/v3/runOpsMigration/resolveInheritedMintKind.server.ts
• apps/webapp/app/v3/runOpsMigration/runOpsMintKind.server.test.ts

🚧 Files skipped from review as they are similar to previous changes (8)

• apps/webapp/app/v3/runOpsMigration/resolveInheritedMintKind.server.test.ts
• apps/webapp/app/runEngine/concerns/idempotencyResidency.server.test.ts
• internal-packages/run-store/src/runOpsStore.routedReadPrimary.test.ts
• apps/webapp/test/engine/triggerTask.test.ts
• apps/webapp/app/v3/services/batchTriggerV3.server.ts
• apps/webapp/test/batchTriggerV3ResidencyInheritance.test.ts
• packages/core/src/v3/isomorphic/runOpsResidency.ts
• packages/core/src/v3/isomorphic/runOpsResidency.test.ts

[devin-ai-integration]

Devin Review found 1 new potential issue.