remember

Executive Summary

This document describes a memory architecture for AI agents, humans, and subsystems sharing one substrate. The substrate stores entities, facts, state-changes, decision traces, and interaction records. Lenses (persistent, role-keyed, DB-resident filters) render the substrate for each consumer. Frames (transient task-keyed bindings) hold the intent, plan, footprint, and suppression state for the work currently in front of a consumer. The architectural identity is one equation: view = render(substrate, lens, frame).

The architecture is orthogonal across three axes. The storage axis (Living Memory, remember, Episodic Continuity) tells you where a fact lives. The interpretation axis (substrate vs lens vs frame) tells you whose view of the substrate it belongs to. The action axis (state-change events, outcome → procedure feedback, intentional non-action) tells you how the substrate evolves under work. Every memory feature gets classified on all three.

The substrate runs on my own development fleet: on create (TypeScript / Node / SQLite), with a second implementation under way on Strata (Python / FastAPI / PostgreSQL + pgvector). Lens contracts, decision traces, the frame primitive, the typed state-change stream, and lens-filtered cross-session retrieval are live; the compounding loop — reflect (reweave, conflict resolution, forgetting) — is in active build, not yet live. The companion landscape and pitch decks cover positioning; this document is the architectural reference.

flowchart TB
  subgraph consumers["Consumers — through a lens (role + jurisdiction)"]
    direction LR
    AG["Agents
bob · alex · tessa …"]
    SUB["Subsystems
watchdog · scheduler"]
    PR["Principal
Andrew"]
  end

  subgraph seam["SLF — the seam · Substrate · Lens · Frame · the only shared code"]
    direction TB
    RENDER["render(substrate, lens, frame) → receipt"]
    PORT["HttpMemoryPort · /v1
context · captures · retrieval-feedback"]
    GRANT["Type-1 grant + issuer-signed receipts
+ capture-envelopes  (enforcement: in build)"]
    RENDER --- PORT --- GRANT
  end

  subgraph create["create — where the work happens"]
    direction TB
    CR["Agent-first dev harness
backlog · sessions · gates · dispatch"]
  end

  subgraph remember["remember — what persists"]
    direction TB
    RM["Substrate: facts, decisions, state-changes,
lens contracts, bi-temporal validity"]
    RF["reflect (the Curator)
reduce · reweave · supersede  (compounding: in build)"]
    RM --- RF
  end

  consumers -->|"act through a lens"| seam
  create -->|"reads context · writes captures"| seam
  seam -->|"/v1"| remember
  remember -.->|"lens-filtered views + receipts"| seam
  seam -.-> create

  classDef building stroke-dasharray:4 4;
  class GRANT,RF building;
      

What's Built and Running

Most of what's below is built and running, used daily on the create platform and SFDC dev workflows, though not all of it. Rows still in active build are marked Planned. The goal is to employ it as a real use case, iterate, and prove out the architecture before any broader rollout.

Memory & Learning Core

Harness Engineering & Governance

remember & Knowledge Presentation

In Progress & Vision

The Three Axes

The architecture spans three orthogonal axes. Confusion between axes is the most common architectural mistake: conflating storage with interpretation, or interpretation with action, produces designs that fight themselves.

Axis 1: Storage (where it lives)

Living Memory holds user/workspace context (preferences, behavioral feedback, project facts). remember holds cross-session, cross-project learnings as bi-temporal claims with decision traces. Episodic Continuity holds session-scoped state: scratchpads (narrative) and per-ticket context (now generalized to frames). Storage is operational; documented in MEMORY-TIERS.md.

Axis 2: Interpretation (whose view of it)

Substrate is the universal storage of state: entities, facts, typed state-changes, relationships, provenance, permissions, history. Bi-temporal validity (valid_from / invalid_at), decision traces, lesson chains, and interaction-typed records all live here. Lens is a persistent consumer-keyed filter declaring entity types loaded by default, relationships in scope, watched state-changes, RBAC capabilities, cross-lens read scope, and expiration policy. Lenses are DB-resident (lens_contracts + lens_preferences + consumer_lens_assignments + permission_groups + permissions). Frame is a transient task-keyed binding holding intents (canonical 10), next_steps with assigned consumers, footprint, surfaced substrate refs, and suppression state. Frames are also DB-resident; scratchpads carry frame_id at creation.

Axis 3: Action (how it evolves)

Substrate writes emit typed state-change events with before/after diffs, lens_id and frame_id annotation, and evidence references. Outcomes (eval results, gate failures, learning-proposal status) flow back to procedure (SKILL.md mutations) through an approval-gated mutator that records the mutation as a state-change. Triggers (hooks, watchdog, scheduler, evaluator) emit explicit act / wait / escalate / no-op decisions with rationale. Non-action is a first-class action when it is intentional and persisted, and silent failure is structurally distinguishable from intentional restraint.

Dreams: retrospective synthesis (designed, in build). The action axis is designed to include scheduled retrospections inspired by Anthropic's Managed Agents Dreams primitive (research preview, 2026-04). As designed, Dreams — run by the Dreamweaver agent — would run asynchronously over a rolling window of state-changes, scratchpad archives, decision traces, and the docs corpus, surfacing duplicate facts to merge, contradicted facts to retire, stale entries to age out, and skill-improvement proposals based on outcome patterns. Anthropic's version is developer-triggered and adoption-gated by hand; the local design extends that framing with scheduled cadence on the local-LLM fleet (mini1/2/3, studio1) and an automated adoption gate (per-proposal-type confidence and risk thresholds, mandatory approval-queue routing for exceptions). It has not yet run autonomously. When it does, output flows through the same approval-gated mutator as every other substrate change, with provenance tagging so subsequent dreams skip their own prior outputs.

flowchart TB
  subgraph storage["Storage Axis"]
    LM["Living Memory
user / workspace"]
    AL["remember
cross-session learnings"]
    EC["Episodic Continuity
scratchpads + frames"]
  end
  subgraph interp["Interpretation Axis"]
    S["Substrate
facts, state-changes,
decision traces, interactions"]
    L["Lens
DB-resident, RBAC,
consumer-keyed"]
    F["Frame
DB-resident, transient,
task-keyed"]
  end
  subgraph action["Action Axis"]
    SC["Typed state-changes
before/after + lens_id + frame_id"]
    OP["Outcome → Procedure
approval-gated mutator"]
    NA["Intentional non-action
act / wait / escalate / no-op"]
    RC["render(...) → receipt
signed disclosure, first-class"]
  end
  storage -.->|"writes through"| interp
  interp -.->|"evolves via"| action
  action -.->|"persisted in"| storage
      

The Learning Loop (Step-by-Step)

Every agent session follows the same lifecycle. The loop is the same whether the agent is working on a core platform ticket or a Salesforce development task.

sequenceDiagram
    participant Agent as Agent (in create)
    participant API as create API
    participant Seam as SLF /v1 seam
    participant Store as remember substrate
    participant Reflect as reflect (Curator)

    Agent->>API: POST /sessions (register)
    API-->>Agent: Session ID + scratchpad
    Agent->>Seam: GET /v1/context (lens, frame)
    Seam->>Store: query substrate (lens-filtered)
    Store-->>Seam: lessons + decision traces
    Seam-->>Agent: context payload + receipt
    Note over Agent: Agent works · updates scratchpad every 3-5 actions
    Agent->>Seam: POST /v1/captures (capture-envelope)
    Seam->>Store: write (Type-1 grant · signed receipt)
    Store->>Reflect: queue compounding  [in build]
    Reflect->>Store: reduce → reweave → supersede  [in build]
    Note over Store: next /v1/context returns these
      

1. Register session. Agent identifies itself, declares what it's working on. System creates or loads a scratchpad (per-task state). Per Cornelius: the vault encounter IS identity. The agent becomes itself by reading what previous sessions left behind.
2. Load context (getContext). System retrieves relevant lessons and precedent, scoped by workflow and task similarity, within a token budget (Cherny's context engineering). Optionally includes remember index and rendered claims.
3. Do the work. Agent uses skills, tools, and files. Updates scratchpad with state, decisions, and progress. Following PAL's insight: agents produce structured artifacts (code, scratchpads, decision traces) in place of free-form prose. Code-as-reasoning.
4. End session with reflection. Agent submits: summary of what was done, structured reflection (what was learned, what to watch for), and optionally guardrail suggestions, knowledge capture, or warnings for future sessions.
5. System writes and digests. Lessons and decision traces are stored (workflow-scoped). The reflect pipeline processes them asynchronously: extracting atomic claims, finding connections, updating existing knowledge, validating, and archiving source material.
6. Next session benefits. The next agent's getContext returns these lessons. The loop compounds. Per Karpathy's LLM OS: the knowledge store is the file system of the agent operating system; each session is a process that reads from and writes to it.

The reflect Pipeline

Raw knowledge entering the system is not yet useful. It is food awaiting digestion before it becomes capability. The reflect pipeline transforms it. Each phase runs in isolation with fresh context to prevent contamination between steps.

flowchart LR
    R["Reduce
raw → atomic claims"]
    RE["Reflect
find connections"]
    RW["Reweave
update existing · in build"]
    V["Verify
schema + dedup · in build"]
    A["Archive
mark processed"]
    R --> RE --> RW --> V --> A
    classDef building stroke-dasharray:4 4;
    class RW,V building;
      

How Retrieval Works

Retrieval is purpose-aware and token-budgeted. The same entity returns different claim subsets depending on the consumer's workflow and task.

flowchart TB
    Q["Agent calls /v1/context
(lens, frame, scopes)"]
    F["Filter by lens + scope
(project → team → framework → org)"]
    R["Rank: priority × recency
× tag match × value-per-token"]
    B["Apply token budget"]
    BP["Berrypicking: multi-pass
(reassess, refine, re-query)"]
    A["Assemble: lessons + precedent
+ (optional) remember index + receipt"]
    Q --> F --> R --> B --> BP --> A
      

The Knowledge Graph

The graph is entity-centric: it models the real things the organization cares about and the relationships between them. Entities are the nouns of institutional knowledge.

flowchart LR
    subgraph SRC["Sources (producers)"]
        direction TB
        s1["Agent reflections (create)"]
        s2["Research"]
        s3["Conversations"]
        s4["Events"]
    end

    subgraph INC["SLF capture-envelope"]
        c1["entity + claim + source +
confidence + timestamp + author +
lens_id + frame_id + actor-chain"]
    end

    subgraph KG["remember substrate"]
        direction TB
        e["Entities"]
        r["Relationships"]
        kc["Claims"]
        b["Backlinks"]
        e --- r --- kc --- b
    end

    subgraph OC["render → receipt"]
        o1["Lens-filtered context
+ signed receipt"]
    end

    subgraph CON["Consumers (via lens)"]
        direction TB
        a1["Agents"]
        a2["Dashboards"]
        a3["Decision support"]
        a4["Search"]
    end

    SRC --> INC --> KG --> OC --> CON
      

Consumers (Lenses in Use)

Agents are one consumer class. Humans are another. Subsystems (watchdog, scheduler, dispatcher) are a third. All read the same substrate through different lenses. A consumer can wear more than one lens; multi-lens permissions UNION (wearing more lenses can only widen scope, never restrict it). Andrew, as Principal, can wear any lens to view the work through that role's filter.

Skills are workspace-wide capabilities. A loose-coupling join records which lens each skill is primary for, which skills bridge between lenses (Bea's R×S to Backlog hands work from Analyst to Steward), and which are lens-agnostic (the career-coach skill operates outside any analyst/developer lens). The lens prioritizes and surfaces skills; permissions gate invocation. Skills don't gate against the active lens; carrying a skill while wearing the “wrong” lens is permitted, just not surfaced by default.

Not every agent should learn. The substrate still serves four levels of agent complexity, distinguished by their relationship to knowledge: stateless tools (run, return, forget), knowledge consumers (read substrate, no writes), scoped learners (write within a single lens), and full-metabolism agents (write across substrate with reflection and decision-trace contribution). The 65/20/10/3 distribution still holds. The lens layer cuts orthogonally to autonomy: a stateless tool can be Developer-lensed (formatter); a full-metabolism agent can be Architect-lensed (alex). Lens determines scope; autonomy determines write capacity.

Honest Challenges

We pressure-tested the architecture against 30+ external sources across 10+ sessions. Seven validated strengths. Five remaining gaps.

Validated strengths (protect these)

Remaining gaps (honest assessment)

Architecture Vision

The substrate, lenses, and frames are real and running; the compounding-learning loop is the current build. The vision is what comes next: the architectural shape carrying forward to additional consumer classes (enterprise, customer-facing), additional input sources (interaction memory beyond the local agent loop), and longitudinal compounding measured against the right metrics, the ones suited to the substrate rather than retrieval-speed benchmarks borrowed from RAG.

flowchart TB
    subgraph now["Operational today"]
        S["remember substrate
facts, state-changes,
decision traces"]
        L["Lens contracts
(7-lens roster, RBAC)"]
        F["Frame primitive"]
        SLF["SLF seam
/v1 · Type-1 grant · receipts"]
        S --> L --> F
        S --> SLF
    end
    subgraph building["In build"]
        LOOP["Compounding loop
reweave · conflict resolution · forgetting"]
        GATE["Gate-authoring-at-ingest
(signed gates on write)"]
        GRANTS["Grant Types 2–7
+ counter-signed receipts"]
    end
    subgraph extending["Extending"]
        CR["create emits receipts
(seam in production)"]
        IM["Interaction memory
(meetings, calls — opt-in)"]
        LP["Lens packs
(Sales, Success, Support, Exec)"]
    end
    S -.-> LOOP
    SLF -.-> GATE
    SLF -.-> GRANTS
    SLF -.-> CR
    S -.-> IM
    L -.-> LP
      

Three directions of growth

The Knowledge Interchange Layer

Any source publishes knowledge into the substrate through one contract: {entity, claim, source, confidence, timestamp, author, lens_id, frame_id}. The lens and frame annotations are what make the contract substrate-shaped: every write knows whose view produced it and which task it served. A well-formed claim is a capability the agent gains on load. The framework grows by accretion.

The SLF protocol →  ·  create: the harness →  ·  remember: the strategy deck →  ·  Back to portfolio