Skip to content

feat(skill-memory): per-skill cross-session recall + historian auto-extraction#181

Draft
iceteaSA wants to merge 4 commits into
cortexkit:masterfrom
iceteaSA:skill-memory-pr
Draft

feat(skill-memory): per-skill cross-session recall + historian auto-extraction#181
iceteaSA wants to merge 4 commits into
cortexkit:masterfrom
iceteaSA:skill-memory-pr

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds skill-memory — per-skill "motor memory" that gives a skill cross-session recall of its own hard-won lessons (gotchas, discoveries, fixes, workflow steps). When a skill's SKILL.md declares skill-memory: { enabled: true }, accumulated notes for that skill surface automatically in a <skill-memory> block appended to the skill tool's result on every load — and, with the historian extension, the historian writes those notes automatically during compaction (no agent action required).

It's fully opt-in per skill and cache-safe by construction: the block rides the tool-result tail (conversation), never the cached system/m[0] prefix, so it can't bust the prompt cache.

How it works

  • Transparent recall — three-hook augmentation: tool.definition advertises an intent param; tool.execute.before stashes the intent (bounded TTL); the after-hook parses the skill's Base directory, reads its SKILL.md frontmatter, and formats the recall block. Lands in the tool RESULT (cache-safe).
  • Write pathsctx_skill_note (agent-authored) and the historian (auto-extracted). Both dedup on a normalized hash.
  • Intent-scoped ranking — a recall cascade: model-matched embeddings → cosine blend over intent_embedding + delta_embedding (relevance/recency/hit weights tunable per skill via ranking_* frontmatter); FTS5 fallback over a content-linked skill_memory_fts vtable; flat recency×hit fallback.
  • Historian auto-extraction — the historian sees TC: skill(<name>) markers in its chunk and emits a <skill_observations> block; both the OpenCode and Pi runners promote those post-commit as global notes (project_identity='*', source_type='historian'), recallable from any project.

Review units (4 commits)

The branch is organized into four coherent, reviewable phases:

  1. P1 — transparent per-skill recall: skill_memory table migration, the three-hook augmentation, flat recall + storage, ctx_skill_note/ctx_skill_recall tools, opt-in distill-skill-memory dreamer task, TUI/ctx-status stats, docs.
  2. P2 — embeddings + intent-scoped recall: delta_embedding + recall_count columns + skill_memory_fts FTS5 vtable; embed-on-write; the cosine/FTS recall cascade; a programmatic (no-LLM) reembed pre-step.
  3. P3a — historian-extraction foundation: the TC: skill(<name>) marker keystone; origin_project + source_type columns; global-tier notes unified under project_identity='*' (collision-merge); partitionKey routing.
  4. P3b — historian auto-extraction pipeline: prompt emits <skill_observations>, parser, validated-result threading, both runners promote via the shared promoteSkillObservations helper, plus an initializeDatabase self-heal net (re-creates skill_memory + ensureColumn so an upgraded DB recovers even if a migration row is lost).

Schema / migrations

Three migrations (numbered after the current master ceiling): skill_memory table, then delta_embedding/recall_count/FTS, then origin_project/source_type/* unification. LATEST_SUPPORTED_VERSION bumped in lockstep (the schema-version-fence test enforces it). Migration bodies are ensureColumn/IF NOT EXISTS idempotent.

Testing

Full plugin + Pi suites pass (one unrelated pre-existing full-suite ordering flake in tui-config.test.ts that passes in isolation), tsc clean both packages, lint clean, build produces all bundles. Dedicated coverage for the migrations (coexistence + fence), recall rungs, the tools, FTS triggers/backfill, the '*' collision-merge, and the historian promotion path on both runners. Verified working live: the historian auto-extraction writes genuine source_type='historian' notes, embeddings populate, and the read-side recall_count increments on surfacing.

Notes for reviewers

  • Migration version numbers are placeholders relative to this fork's base — happy to renumber to whatever slots are free at merge time.
  • The four commits are independently meaningful; if you'd prefer this as stacked PRs (P1 / P2 / historian), I can split it.

View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


Summary by cubic

Adds per-skill cross-session memory with opt-in via SKILL.md, appending a cache-safe <skill-memory> block to each skill tool RESULT. Adds intent-scoped recall with embeddings, historian auto-extraction into global notes, an opt-in distill-skill-memory dreamer task, and migrations to v52 with an init-time self-heal.

  • New Features

    • Per-skill recall: ctx_skill_note writes notes; ctx_skill_recall fetches without reloading. The skill tool advertises an optional intent; a before-hook stashes it and an after-hook formats recall into the tool RESULT (cache-safe).
    • Ranking + embeddings: cosine blend over intent_embedding + delta_embedding with FTS5 fallback; read-side recall_count. Programmatic re-embed backfills NULL/stale vectors before the opt-in distill-skill-memory pass. Status/TUI show per-project totals and pinned counts.
    • Historian auto-extraction: TC: skill(<name>) markers; the historian emits <skill_observations>, the parser extracts them, and both runners promote them as global '*' notes (source_type='historian').
    • Dreamer + config/UI: added distill-skill-memory (off by default; always-eligible gate) with a read-only corpus health report; CLI setup, dashboard, and schema support; guidance added to the main prompt.
    • Schema/migrations/self-heal: added skill_memory table + FTS vtable and columns (delta_embedding, recall_count, origin_project, source_type); LATEST_SUPPORTED_VERSION set to 52 and initializeDatabase now re-creates the table/columns/FTS/triggers if missing.
  • Bug Fixes

    • Moved buildHiddenAgentConfig to a non-entry module so only the default export remains; prevents opencode 1.17 from treating it as a plugin factory and breaking hook/tool registration.

Written for commit dc83db6. Summary will update on new commits.

Review in cubic

Tehan added 3 commits June 24, 2026 22:24
Per-skill "motor memory": when a skill's SKILL.md declares
`skill-memory: { enabled: true }`, accumulated gotchas/discoveries/fixes/
workflow-steps surface in a <skill-memory> block appended to the skill
tool's RESULT on every load (cache-safe — rides the tool-result tail).
Agents write back via ctx_skill_note; ctx_skill_recall is the explicit
companion to the transparent after-hook.

- migration: skill_memory table (per-skill; tier project/global; UNIQUE on
  skill_id/tier/project_identity/normalized_hash) + lookup indexes.
- three-hook augmentation: tool.definition advertises an `intent` param;
  tool.execute.before stashes intent (bounded TTL); after-hook parses the
  skill's Base directory, reads SKILL.md frontmatter, formats the block.
- flat recency×hit recall + storage layer; ctx_skill_note / ctx_skill_recall.
- opt-in distill-skill-memory dreamer task; agent-prompt guidance; TUI/ctx-status stats.
- docs: ARCHITECTURE / STRUCTURE / CONFIGURATION / README.
Upgrade recall from flat recency×hit to a multi-rung cascade: intent +
model-matched embeddings → cosine blend across intent_embedding +
delta_embedding (relevance/recency/hit weights tunable per skill via
ranking_* frontmatter); intent + no model match → FTS5 fallback over the
content-linked skill_memory_fts vtable; empty → flat fallback.

- migration: delta_embedding + recall_count columns + skill_memory_fts FTS5 vtable.
- embed-on-write in insertSkillMemoryNote; delta-only semantic dedup.
- programmatic, no-LLM reembed pre-step for the distill-skill-memory dreamer task.
- read-side recall_count (distinct from write-side hit_count).
- canonical vector serde + dedup/ranking/FTS query helpers.
…ication (P3a)

Foundation for the historian to auto-capture skill notes cross-project.

- surface the skill name in the historian chunk as a `TC: skill(<name>)`
  marker (the keystone — the tool input name was previously dropped).
- migration: origin_project + source_type columns; unify global-tier notes
  under project_identity='*' (collision-merge) so a global note is one row
  recallable from any repo.
- partitionKey helper routes global write/recall/reembed/stats through '*';
  recall reads global-tier from '*' (cross-project); reembed sweeps '*'.
Close the loop so the historian writes skill notes during compaction
without an agent volunteering ctx_skill_note.

- historian prompt emits a <skill_observations> block; parser extracts it;
  threaded through the validated historian result.
- both runners (OpenCode + Pi) promote skill observations post-commit via
  the shared promoteSkillObservations helper, gated by
  promotionActive && !discardedLast, writing global '*' notes with
  source_type='historian'.
- self-heal net: initializeDatabase re-creates skill_memory + ensureColumn
  so an upgraded DB recovers even if a migration row is lost.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant