perf(trigger): cap concurrency on background DB tasks#5231
perf(trigger): cap concurrency on background DB tasks#5231TheodoreSpeaks wants to merge 2 commits into
Conversation
|
@greptile review |
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Schedule execution default drops 50 → 30 ( Reviewed by Cursor Bugbot for commit c223eef. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryThis PR caps concurrency for DB-heavy Trigger.dev background work. The main changes are:
Confidence Score: 5/5The changed flow looks mergeable after clarifying queue identity and updating the schedule-budget test.
apps/sim/background/workflow-execution.ts and apps/sim/lib/workflows/schedules/execution-limits.ts Important Files Changed
|
| export const workflowExecutionTask = task({ | ||
| id: 'workflow-execution', | ||
| machine: 'medium-1x', | ||
| queue: { |
There was a problem hiding this comment.
When queue has a concurrencyLimit but no explicit name, these new high-volume tasks depend on Trigger.dev's implicit queue identity. If nameless queues resolve to a shared default queue, workflow, webhook, resume, and existing nameless cleanup jobs can throttle each other, so a webhook burst can delay unrelated workflow or resume runs instead of applying independent per-task caps.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Greptile SummaryThis PR caps Trigger.dev background task concurrency for heavier DB-bound work. The main changes are:
Confidence Score: 5/5This looks safe to merge after checking how the new env overrides are injected for Trigger workers.
apps/sim/background/concurrency-limits.ts Important Files Changed
Reviews (2): Last reviewed commit: "perf(trigger): cap concurrency on backgr..." | Re-trigger Greptile |
| export const WORKFLOW_EXECUTION_CONCURRENCY_LIMIT = envNumber( | ||
| env.WORKFLOW_EXECUTION_CONCURRENCY_LIMIT, | ||
| 75, | ||
| { min: 1, integer: true } |
There was a problem hiding this comment.
Module Load Freezes Env Defaults
When these constants are evaluated before the Trigger worker has the new env vars in process.env, envNumber falls back to 75 and the override is never read again. In deployments where Trigger-specific env vars are only injected through the Trigger project/runtime settings, lowering WORKFLOW_EXECUTION_CONCURRENCY_LIMIT would be silently ignored and the task can keep running at the default cap.
Summary
apps/sim/background/concurrency-limits.tsdefines env-overridable caps via the existingenvNumberpattern:workflow-execution75 (WORKFLOW_EXECUTION_CONCURRENCY_LIMIT),webhook-execution75 (WEBHOOK_EXECUTION_CONCURRENCY_LIMIT),resume-execution50 (RESUME_EXECUTION_CONCURRENCY_LIMIT).SCHEDULE_EXECUTION_CONCURRENCY_LIMIT).env.ts.This is a strict reduction in peak concurrency — the three tasks were previously unbounded.
Validated against 30d of prod run data
SCHEDULE_WORKFLOW_ENQUEUE_LIMIT = LIMIT × 2drops 100→60 in tandem.Type of Change
Testing
Tested manually —
bun run lint(biome) clean,bun run check:api-validation:strictpasses,tsc --noEmitclean on changed files. No API boundaries touched.Checklist