feat: MCP server refactor with fine-grained tools and incremental update support#66
feat: MCP server refactor with fine-grained tools and incremental update support#66mambo-wang wants to merge 28 commits into
Conversation
🤖 Generated with [Qoder][https://qoder.com]
- Add IDE_DRIVEN_GUIDE.md with complete walkthrough for using CodeWiki with AI IDEs (CodeBuddy, Cursor, Claude Desktop) via MCP - Update README with IDE-Driven Mode section and navigation link
- 8 个模块文档（Agent 工具、CLI 工具、CLI 核心、MCP 服务、依赖分析器、共享配置、前端服务、后端核心） - 仓库总览 overview.md - 模块聚类树 module_tree.json - 全部文档含 Mermaid 架构图，语法校验通过
- Add _detect_changes() with git diff + mtime dual-strategy detection - Add _find_affected_modules() to map changed files to affected modules - analyze_repo now returns a 'changes' field with affected/cascade modules - Decouple codewiki/__init__.py from CLI imports for lightweight MCP startup - Update skill and IDE_DRIVEN_GUIDE.md with incremental update docs
Previously, CLIDocumentationGenerator never received or forwarded the git commit SHA, so metadata.json always had commit_id: null. This made --update fall back to full regeneration every time. Now the commit hash is obtained before generator creation and threaded through to the backend DocumentationGenerator, matching the behavior already present in Web mode (background_worker.py).
Square logo for GitHub repo avatar and wide banner for README header. Design follows the blue-purple-green gradient palette from the original CodeWiki framework diagram, with a red CN badge for branding.
Square logo corners and banner surrounding area are now transparent instead of white/light-gray, suitable for any background color.
Remove transparent padding around the rounded rectangle by filling corners with a matching dark navy gradient, making the banner a solid rectangle.
- Add _detect_changes() with git diff + mtime dual-strategy detection - Add _find_affected_modules() to map changed files to affected modules - analyze_repo now returns a 'changes' field with affected/cascade modules - Decouple codewiki/__init__.py from CLI imports for lightweight MCP startup - Update skill and IDE_DRIVEN_GUIDE.md with incremental update docs
Previously, CLIDocumentationGenerator never received or forwarded the git commit SHA, so metadata.json always had commit_id: null. This made --update fall back to full regeneration every time. Now the commit hash is obtained before generator creation and threaded through to the backend DocumentationGenerator, matching the behavior already present in Web mode (background_worker.py).
- Reduce component_index from 500 to 100 items per page (max 200), drop depends_on from each entry (available via read_code_components) - Add offset/limit params to analyze_repo for pagination - Add list_components tool for browsing components without re-analysis - Reduce leaf_nodes from 100 to 50 - Remove IDE rule files, consolidate into skill files
… hang - Wrap synchronous tool handlers in asyncio.to_thread() to avoid blocking the event loop (analyze_repo excluded — Tree-sitter C extensions are not thread-safe) - Disable mermaid-py validation by default (set MERMAID_VALIDATE=1 to enable), add 15s timeout to prevent indefinite hangs
- Fix shell injection in view_repo_file: replace shell=True subprocess with pathlib iteration - Add path traversal guards in view_repo_file, write_doc_file, edit_doc_file (reject paths escaping repo/output dir) - Add threading.Lock to SessionStore for concurrent access safety - Cap max sessions to 10, evict oldest when full - Cap read_code_components to 50 IDs per call - Cap edit history to 20 entries per file - Store edit history as native dict instead of JSON string - Fix undo to run Mermaid validation after reverting content - Fix Mermaid validation to report "skipped" instead of false success - Fix Mermaid timeout to warn instead of silent pass - Fix pagination hint to point to list_components instead of analyze_repo - Clamp offset to non-negative in _build_component_index - Add smoke test covering all critical paths (25 assertions)
- Reduce _MAX_RESPONSE_LEN 32000→24000, _MAX_COMPONENTS_PER_CALL 50→20 - Add per-component source truncation at 8000 chars - Write metadata.json (git commit_id + timestamp) on close_session to enable incremental update detection on next analyze_repo - Update smoke test assertions to match new caps
- Rewrite MCP 服务.md: add list_components tool, thread-safe SessionStore, path traversal guards, incremental update mechanism, multi-layer truncation - Update 后端核心.md: document mermaid validation degradation strategy - Update overview.md: tool count 9→10, MCP module component count 27→38 - Refresh module_tree.json with new MCP components
Replace large data transmission through stdio MCP protocol with
file-based side channels, enabling support for larger codebases.
Key changes:
- Add SessionWorkspace for per-session disk workspace management
- Write component index, leaf nodes, source files to {repo}/.codewiki/sessions/
- Remove list_components and view_repo_file tools (agent uses native file reading)
- Remove all truncation/pagination limits from MCP responses
- Fix Windows GBK encoding issues with explicit utf-8 encoding
- Update skill SKILL.md to v2.0.0 reflecting new 8-tool architecture
- Regenerate wiki docs with updated module structure (19 docs)
anhnh2002
left a comment
anhnh2002
left a comment
There was a problem hiding this comment.
Nice refactor overall, the session/tools split is clean and the CLI commit_id fix is correct. A few things to address before merge.
Blockers: the mcp SDK is still not a declared dependency (the pyproject change only registers packages), and view_repo_file shells out with an interpolated path which breaks on spaces and allows command injection.
Should-fix: path traversal on agent-supplied filenames in the read/write tools, and the incremental-update feature only works for CLI-generated docs since the MCP flow never writes metadata.json. Details inline.
- Add mcp>=1.0.0 to pyproject.toml dependencies (fixes ModuleNotFoundError on fresh install) - Add explicit utf-8 encoding to FileManager file operations
|
✅ Issue 2 — code_reader.py 命令注入：当前的 code_reader.py 已经彻底重写，没有任何 subprocess 或 shell=True 调用，全部用纯 Python 的 Path 操作写文件，命令注入漏洞已消除。 overall: ✅ mcp SDK 已声明依赖（刚加的） |
anhnh2002
left a comment
anhnh2002
left a comment
There was a problem hiding this comment.
Nice work on the MCP refactor and the incremental-update support, the file side-channel design is clean. Two things to sort before merge.
First, unrelated content from the fork needs to come out: CodeWiki介绍.md and img/logo-banner.png are unrelated, and IDE_DRIVEN_GUIDE.md + skills/codewiki-wiki-generator/SKILL.md hardcode the fork name CodeWiki-CN and a mambo-wang/CodeWiki-CN clone URL that should point at upstream. The guide also references .codebuddy/.../RULE.mdc and .cursorrules rule files that aren't in the PR.
Second, a few code issues worth fixing: the off-by-start line numbers in edit_doc_file snippets, the Windows path-separator mismatch in mtime detection, the substring over-matching in _find_affected_modules, the missed staged changes in git detection, and the Mermaid validation being off by default while the docs advertise it as automatic. Details inline.
- Remove unrelated files (CodeWiki介绍.md, img/logo-banner.png)
- Replace fork-specific references (CodeWiki-CN) with upstream (CodeWiki)
in IDE_DRIVEN_GUIDE.md and SKILL.md
- Fix off-by-start line number calculation in edit_doc_file snippet window
- Use .as_posix() for mtime path detection to fix Windows path separators
- Fix _find_affected_modules substring over-matching with path-based matching
- Use index.diff('HEAD') to capture staged changes (not just unstaged)
- Enable Mermaid validation by default to match documentation promises
- Add SHA1 hash suffix to _safe_filename to prevent component ID collisions
|
全部完成，已推送到 PR #66。修改汇总： 删除无关文件：CodeWiki介绍.md 和 img/logo-banner.png 清除 fork 引用：IDE_DRIVEN_GUIDE.md 和 SKILL.md 中所有 CodeWiki-CN 替换为上游 CodeWiki 代码修复（6处）：
|
anhnh2002
left a comment
anhnh2002
left a comment
There was a problem hiding this comment.
Thanks for the thorough revision — nearly all prior feedback is correctly addressed (mcp dependency, path-traversal guards, shell-injection removal, SHA1 filename suffix, as_posix()/index.diff("HEAD")/path-boundary matching in incremental detection, thread-safe SessionStore, removal of unrelated assets, and the CodeWiki-CN scrub).
One previously-flagged issue is marked resolved but not actually fixed: the snippet line-number labels in edit_doc_file (both the str_replace and insert branches) still use i + start + 1, which double-counts start. See the two inline comments — each is a one-character fix to i + 1. The window-end change in the last commit was a separate (valid) fix, but the label the original review pointed at is unchanged.
Two minor, non-blocking nits for follow-up:
_write_generation_metadatainserver.pywritesmetadata.jsonwithoutencoding="utf-8", inconsistent with the utf-8 hardening applied elsewhere in this PR.- Incremental update now depends on
metadata.json, which is written only onclose_session; if a session ends withoutclose_session, the nextanalyze_reposilently falls back to full analysis. Worth a one-line note in the SKILL/guide.
The feature itself is well worth merging — the fine-grained IDE-driven tool set and incremental update support are real improvements. Just the line-number fix before merge.
| lines = new_content.split("\n") | ||
| start = max(0, replacement_line - 4) | ||
| end = min(len(lines), start + new_str.count("\n") + 9) | ||
| snippet = "\n".join(f"{i + start + 1:6}\t{lines[i]}" for i in range(start, end)) |
There was a problem hiding this comment.
The snippet line numbers are still off by start. i already runs over absolute indices from range(start, end), so the label should be i + 1, not i + start + 1. With start=10 the first line gets labeled 21 instead of 11; it's only correct when start == 0 (edits near the top of the file).
The earlier fix corrected the window end bound, but the label expression flagged in the previous review still double-counts start. The agent reads this snippet back to choose its next insert_line, so a wrong number can misdirect the following edit.
| snippet = "\n".join(f"{i + start + 1:6}\t{lines[i]}" for i in range(start, end)) | |
| snippet = "\n".join(f"{i + 1:6}\t{lines[i]}" for i in range(start, end)) |
|
|
||
| start = max(0, insert_line - 4) | ||
| end = min(len(lines), start + len(new_str_lines) + 8) | ||
| snippet = "\n".join(f"{i + start + 1:6}\t{lines[i]}" for i in range(start, end)) |
There was a problem hiding this comment.
Same off-by-start bug in the insert branch — the label should be i + 1, not i + start + 1.
| snippet = "\n".join(f"{i + start + 1:6}\t{lines[i]}" for i in range(start, end)) | |
| snippet = "\n".join(f"{i + 1:6}\t{lines[i]}" for i in range(start, end)) |
2 participants
Summary
--update/commit_id) to MCPanalyze_repo, so only changed modules are re-analyzedcommit_idpassthrough tometadata.jsonin CLI mode for--updatesupportpyproject.tomlTest plan
python -m codewiki.mcp.serverand all new tools are listedanalyze_repotool, confirm session is created and dependency graph is builtanalyze_repoagain withcommit_idto verify incremental update works correctly--updateflag passes commit_id to metadata.json as expectedgenerate_docs,get_module_tree) still function