Visualize context window usage in Ask Sourcebot#1370
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (6)
WalkthroughAdds context-window resolution to chat flows, carries that value through message metadata, and renders a usage gauge in chat details. The changelog also notes the new gauge. ChangesContext window usage gauge
Sequence Diagram(s)sequenceDiagram
participant ChatRoute
participant AskCodebase
participant ResolveContextWindow
participant CreateMessageStream
participant DetailsCardComponent
ChatRoute->>ResolveContextWindow: resolveContextWindow(languageModelConfig)
AskCodebase->>ResolveContextWindow: resolveContextWindow(languageModelConfig)
ResolveContextWindow-->>ChatRoute: contextWindow
ResolveContextWindow-->>AskCodebase: contextWindow
ChatRoute->>CreateMessageStream: createMessageStream(..., contextWindow)
AskCodebase->>CreateMessageStream: createMessageStream(..., contextWindow)
CreateMessageStream-->>DetailsCardComponent: message-metadata.contextWindow
DetailsCardComponent->>DetailsCardComponent: derive usage and render gauge
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Line 15: Update the changelog entry for the context-window usage gauge to
follow the enterprise-only convention by adding the [EE] prefix, and replace the
placeholder PR reference with the real GitHub PR number/link. Keep the wording
otherwise the same, and make sure the entry still clearly describes the Ask
Sourcebot chat details gauge and its models.dev-based window sizing.
In `@packages/web/src/features/chat/modelContextWindow.server.ts`:
- Around line 59-73: The loadCatalog caching flow currently drops the previously
cached ModelsDevCatalog on a failed refresh and immediately retries every
request, which can block chat sends on the request path. Update loadCatalog so a
TTL-triggered fetch failure preserves and returns the last-known-good catalog
instead of replacing it with null, and add a short negative-cache/backoff window
for repeated fetchCatalog failures. Keep the retry behavior bounded, but avoid
clearing a usable catalog or reattempting on every resolveContextWindow call.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 808a7227-a756-49ec-9fad-14d26485aa46
📒 Files selected for processing (8)
CHANGELOG.mdpackages/web/src/app/api/(server)/ee/chat/route.tspackages/web/src/ee/features/chat/agent.tspackages/web/src/ee/features/chat/components/chatThread/detailsCard.tsxpackages/web/src/ee/features/mcp/askCodebase.tspackages/web/src/features/chat/modelContextWindow.server.tspackages/web/src/features/chat/modelContextWindow.test.tspackages/web/src/features/chat/types.ts
| if (percent >= CONTEXT_USAGE_YELLOW_PERCENT) { | ||
| return "text-yellow-500"; | ||
| } | ||
| return "text-[#6cb38f]"; |
Contributor
There was a problem hiding this comment.
to confirm, does this look good in light mode?
brendan-kellam
approved these changes
Jun 26, 2026
Resolve each model's context window from the models.dev catalog (already fetched by the setup wizard) and bake it into chat message metadata. The Details card now renders a usage gauge from the latest step's input tokens. Models with no catalog entry (openai-compatible/self-hosted) fall back to the existing raw token count.
createMessageStream's MCP/programmatic caller omitted contextWindow, so chats created via ask_codebase rendered the Details card without a usage gauge even for catalogued models — unlike the same chat created from the web API. Resolve it from the already-available languageModelConfig so the gauge is deterministic per model, not per entry point.
Replace the horizontal bar with a circular ring showing the percentage inside and the used / total token counts beside it. The arc and percentage are colored by usage — green below 70%, yellow from 70%, red from 90%.
Shrink the ring and move the percentage beside it, reading "<percent>% of <total>" instead of a number-in-ring. The arc and percentage stay colored by usage (green/yellow/red).
Reduce the ring to 14px, switch the track to a solid palette gray (the theme tokens lack an alpha channel, so /opacity on them was ignored and the track rendered at full brightness), and use a desaturated sage green for the in-range percentage.
Also drop the gauge's yellow/red thresholds from 70/90 to 50/80 so the ring shifts color earlier as the context fills.
Real PR number to be filled in once the PR is opened.
Previously a failed fetch reset the cache to null, so every chat send re-attempted the fetch and blocked up to the 8s timeout during a models.dev outage; a failed TTL refresh also discarded the previously-cached catalog. Switch loadCatalog to stale-while-revalidate: after the first successful load the request path never blocks (it serves the last-known-good catalog and refreshes in the background), failed refreshes keep the cached value, and a 60s negative-cache window bounds retries during an outage.
jsourcebot
force-pushed
the
jminnetian/visualize-context
branch
from
005f251 to
28f61eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We support showing the context usage in cases where we know the context window the model (from models.dev). For self hosted models or if they are behind an API, this will not currently work.
Summary by CodeRabbit