fix(server): treat database timeout errors as client-retryable (503)#22315
Draft
devin-ai-integration[bot] wants to merge 2 commits into
Draft
fix(server): treat database timeout errors as client-retryable (503)#22315devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
Under high concurrency (many deployments polling simultaneously), the connection pool can be exhausted and asyncpg connection attempts time out. Previously these TimeoutError exceptions returned HTTP 500, which clients do not retry. Now they return HTTP 503, enabling automatic client retries with backoff. Added to retryable exceptions: - TimeoutError (asyncpg connection timeout during pool overflow) - sqlalchemy.exc.TimeoutError (pool checkout timeout) - asyncpg.exceptions.TooManyConnectionsError (PostgreSQL max_connections) closes #22314 Co-authored-by: Alexander Streed <alex.s@prefect.io>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Merging this PR will not alter performance
Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #22314
When many deployments poll the server simultaneously, the connection pool can be exhausted and
asyncpg.connect()times out. Previously theseTimeoutErrorexceptions were unrecognized byis_client_retryable_exception(), returning HTTP 500. Clients don't retry 500s, so the user sees noisy error logs and failed requests during transient load spikes.This PR adds three exception types to the retryable set so the server returns 503 instead, enabling automatic client retries with backoff:
Checklist
<link to issue>"mint.json.Link to Devin session: https://app.devin.ai/sessions/68eca37c30704d60a68b1794aadf9a4c