Living Memory & Alexandria

Executive Summary

This document describes a memory architecture for AI agents, humans, and subsystems sharing one substrate. The substrate stores entities, facts, state-changes, decision traces, and interaction records. Lenses (persistent, role-keyed, DB-resident filters) render the substrate for each consumer. Frames (transient task-keyed bindings) hold the intent, plan, footprint, and suppression state for the work currently in front of a consumer. The architectural identity is one equation: view = render(substrate, lens, frame).

The architecture is orthogonal across three axes. The storage axis (Living Memory, Alexandria, Episodic Continuity) tells you where a fact lives. The interpretation axis (substrate vs lens vs frame) tells you whose view of the substrate it belongs to. The action axis (state-change events, outcome → procedure feedback, intentional non-action) tells you how the substrate evolves under work. Every memory feature gets classified on all three.

The substrate runs on my own development fleet: on Forge (TypeScript / Node / SQLite), with a second implementation under way on Strata (Python / FastAPI / PostgreSQL + pgvector). Lens contracts, decision traces, the frame primitive, the typed state-change stream, and lens-filtered cross-session retrieval are live; the compounding-learning loop (reweave, conflict resolution, forgetting) is in active build, not yet live. The companion landscape and pitch decks cover positioning; this document is the architectural reference.

What's Built and Running

Most of what's below is built and running, used daily on the Forge platform and SFDC dev workflows, though not all of it. Rows still in active build are marked Planned. The goal is to employ it as a real use case, iterate, and prove out the architecture before any broader rollout.

Memory & Learning Core

Harness Engineering & Governance

Alexandria & Knowledge Presentation

In Progress & Vision

The Three Axes

The architecture spans three orthogonal axes. Confusion between axes is the most common architectural mistake: conflating storage with interpretation, or interpretation with action, produces designs that fight themselves.

Axis 1: Storage (where it lives)

Living Memory holds user/workspace context (preferences, behavioral feedback, project facts). Alexandria holds cross-session, cross-project learnings as bi-temporal claims with decision traces. Episodic Continuity holds session-scoped state: scratchpads (narrative) and per-ticket context (now generalized to frames). Storage is operational; documented in MEMORY-TIERS.md.

Axis 2: Interpretation (whose view of it)

Substrate is the universal storage of state: entities, facts, typed state-changes, relationships, provenance, permissions, history. Bi-temporal validity columns (valid_at / invalid_at), decision traces, lesson chains, and interaction-typed records all live here. Lens is a persistent consumer-keyed filter declaring entity types loaded by default, relationships in scope, watched state-changes, RBAC capabilities, cross-lens read scope, and expiration policy. Lenses are DB-resident (lens_contracts + lens_preferences + consumer_lens_assignments + permission_groups + permissions). Frame is a transient task-keyed binding holding intents (canonical 10), next_steps with assigned consumers, footprint, surfaced substrate refs, and suppression state. Frames are also DB-resident; scratchpads carry frame_id at creation.

Axis 3: Action (how it evolves)

Substrate writes emit typed state-change events with before/after diffs, lens_id and frame_id annotation, and evidence references. Outcomes (eval results, gate failures, learning-proposal status) flow back to procedure (SKILL.md mutations) through an approval-gated mutator that records the mutation as a state-change. Triggers (hooks, watchdog, scheduler, evaluator) emit explicit act / wait / escalate / no-op decisions with rationale. Non-action is a first-class action when it is intentional and persisted, and silent failure is structurally distinguishable from intentional restraint.

Dreams: retrospective synthesis. The action axis includes scheduled retrospections inspired by Anthropic's Managed Agents Dreams primitive (research preview, 2026-04). Dreams run asynchronously over a rolling window of state-changes, scratchpad archives, decision traces, and the docs corpus, surfacing duplicate facts to merge, contradicted facts to retire, stale entries to age out, and skill-improvement proposals based on outcome patterns. Anthropic's version is developer-triggered and adoption-gated by hand. The local implementation extends Anthropic's framing with scheduled cadence on the local-LLM fleet (mini1/2/3, studio1) and an automated adoption gate: per-proposal-type confidence and risk thresholds with mandatory approval-queue routing for exceptions. Output flows through the same approval-gated mutator as every other substrate change, with provenance tagging so subsequent dreams skip their own prior outputs.

flowchart TB
  subgraph storage["Storage Axis"]
    LM["Living Memory
user/workspace"]
    AL["Alexandria
cross-session learnings"]
    EC["Episodic Continuity
scratchpads + frames"]
  end
  subgraph interp["Interpretation Axis"]
    S["Substrate
entities, facts, state-changes,
decision traces, interactions"]
    L["Lens
DB-resident, RBAC,
consumer-keyed"]
    F["Frame
DB-resident, transient,
task-keyed"]
  end
  subgraph action["Action Axis"]
    SC["Typed state-changes
before/after + lens_id + frame_id"]
    OP["Outcome → Procedure
approval-gated SKILL.md mutator"]
    NA["Intentional non-action
act / wait / escalate / no-op"]
  end
  storage -.->|"writes through"| interp
  interp -.->|"evolves via"| action
  action -.->|"persisted in"| storage
      

The Learning Loop (Step-by-Step)

Every agent session follows the same lifecycle. The loop is the same whether the agent is working on a core platform ticket or a Salesforce development task.

sequenceDiagram
    participant Agent
    participant API as HTTP API
    participant Store as Learning Store
    participant Pipeline as Digestive Pipeline

    Agent->>API: POST /sessions (register)
    API-->>Agent: Session ID + scratchpad

    Agent->>API: GET /sessions/{id}/context
    API->>Store: Query lessons, precedent (workflow-scoped)
    Store-->>API: Relevant lessons + decision traces
    API-->>Agent: Context payload (token-budgeted)

    Note over Agent: Agent works (skills, tools, code)
    Note over Agent: Updates scratchpad every 3-5 actions

    Agent->>API: DELETE /sessions/{id} (end session)
    Note over Agent: Includes: summary, reflection,
lessons, decision traces

    API->>Store: Write lessons + traces (scoped)
    API->>Pipeline: Queue digestive processing
    Pipeline->>Store: Reduce → Reflect → Reweave → Verify → Archive
    Note over Store: Next getContext returns
these lessons
      

1. Register session. Agent identifies itself, declares what it's working on. System creates or loads a scratchpad (per-task state). Per Cornelius: the vault encounter IS identity. The agent becomes itself by reading what previous sessions left behind.
2. Load context (getContext). System retrieves relevant lessons and precedent, scoped by workflow and task similarity, within a token budget (Cherny's context engineering). Optionally includes Alexandria index and rendered claims.
3. Do the work. Agent uses skills, tools, and files. Updates scratchpad with state, decisions, and progress. Following PAL's insight: agents produce structured artifacts (code, scratchpads, decision traces) in place of free-form prose. Code-as-reasoning.
4. End session with reflection. Agent submits: summary of what was done, structured reflection (what was learned, what to watch for), and optionally guardrail suggestions, knowledge capture, or warnings for future sessions.
5. System writes and digests. Lessons and decision traces are stored (workflow-scoped). The digestive pipeline processes them asynchronously: extracting atomic claims, finding connections, updating existing knowledge, validating, and archiving source material.
6. Next session benefits. The next agent's getContext returns these lessons. The loop compounds. Per Karpathy's LLM OS: the knowledge store is the file system of the agent operating system; each session is a process that reads from and writes to it.

The Digestive Pipeline

Raw knowledge entering the system is not yet useful. It is food awaiting digestion before it becomes capability. The digestive pipeline transforms it. Each phase runs in isolation with fresh context to prevent contamination between steps.

flowchart LR
    R["Reduce
Raw → atomic claims"]
    RE["Reflect
Find connections"]
    RW["Reweave
Update existing"]
    V["Verify
Schema + dedup"]
    A["Archive
Mark processed"]
    R --> RE --> RW --> V --> A
      

How Retrieval Works

Retrieval is purpose-aware and token-budgeted. The same entity returns different claim subsets depending on the consumer's workflow and task.

flowchart TB
    Q["Agent calls getContext
(workflow, task, scopes)"]
    F["Filter by workflow + scope
(project → team → framework → org)"]
    R["Rank by: priority × recency
× tag match × value-per-token"]
    B["Apply token budget
(cap total context size)"]
    BP["Berrypicking: multi-pass
(reassess, refine, re-query)"]
    A["Assemble response:
lessons + precedent + (optional) Alexandria index"]

    Q --> F --> R --> B --> BP --> A
      

The Knowledge Graph

The graph is entity-centric: it models the real things the organization cares about and the relationships between them. Entities are the nouns of institutional knowledge.

flowchart LR
    subgraph SRC["Sources"]
        direction TB
        s1["Agent reflections"]
        s2["Research"]
        s3["Conversations"]
        s4["Events"]
    end

    subgraph INC["Input Contract"]
        c1["entity + claim +
source + confidence +
timestamp + author"]
    end

    subgraph KG["Knowledge Graph"]
        direction TB
        e["Entities"]
        r["Relationships"]
        kc["Claims"]
        b["Backlinks"]
        e --- r
        r --- kc
        kc --- b
    end

    subgraph OC["Output Contract"]
        o1["Purpose-aware
context assembly"]
    end

    subgraph CON["Consumers"]
        direction TB
        a1["Agents"]
        a2["Dashboards"]
        a3["Decision support"]
        a4["Search"]
    end

    SRC --> INC --> KG --> OC --> CON
      

Consumers (Lenses in Use)

Agents are one consumer class. Humans are another. Subsystems (watchdog, scheduler, dispatcher) are a third. All read the same substrate through different lenses. A consumer can wear more than one lens; multi-lens permissions UNION (wearing more lenses can only widen scope, never restrict it). Andrew, as Principal, can wear any lens to view the work through that role's filter.

Skills are workspace-wide capabilities. A loose-coupling join records which lens each skill is primary for, which skills bridge between lenses (Bea's R×S to Backlog hands work from Analyst to Steward), and which are lens-agnostic (the career-coach skill operates outside any analyst/developer lens). The lens prioritizes and surfaces skills; permissions gate invocation. Skills don't gate against the active lens; carrying a skill while wearing the “wrong” lens is permitted, just not surfaced by default.

Not every agent should learn. The substrate still serves four levels of agent complexity, distinguished by their relationship to knowledge: stateless tools (run, return, forget), knowledge consumers (read substrate, no writes), scoped learners (write within a single lens), and full-metabolism agents (write across substrate with reflection and decision-trace contribution). The 65/20/10/3 distribution still holds. The lens layer cuts orthogonally to autonomy: a stateless tool can be Developer-lensed (formatter); a full-metabolism agent can be Architect-lensed (alex). Lens determines scope; autonomy determines write capacity.

Honest Challenges

We pressure-tested the architecture against 30+ external sources across 10+ sessions. Seven validated strengths. Five remaining gaps.

Validated strengths (protect these)

Remaining gaps (honest assessment)

Architecture Vision

The substrate, lenses, and frames are real and running; the compounding-learning loop is the current build. The vision is what comes next: the architectural shape carrying forward to additional consumer classes (enterprise, customer-facing), additional input sources (interaction memory beyond the local agent loop), and longitudinal compounding measured against the right metrics, the ones suited to the substrate rather than retrieval-speed benchmarks borrowed from RAG.

flowchart TB
    subgraph now["Operational"]
        S["Substrate
(facts, state-changes,
decision traces, interactions)"]
        L["Lens contracts
(7-lens roster, RBAC)"]
        F["Frame primitive
(intents, footprint, suppression)"]
        D["Dreams
(retrospective synthesis,
local-LLM scheduled)"]
        S --> L
        S --> F
        L --> F
        S -.->|"feeds"| D
        D -.->|"proposes mutations to"| S
    end

    subgraph extending["Extending"]
        IM["Interaction memory
(meeting transcripts, calls,
messaging, opt-in)"]
        LP["Lens packs
(Sales, Success,
Support, On-call, Exec)"]
        CC["Customer-facing surface
(UXF descendant)"]
        MET["Substrate-aligned metrics
(discovery, evolution,
governance, compounding)"]
    end

    S -.->|"new producers"| IM
    L -.->|"productization"| LP
    L -.->|"new consumer class"| CC
    now -.->|"validated by"| MET
      

Three directions of growth

The Knowledge Interchange Layer

Any source publishes knowledge into the substrate through one contract: {entity, claim, source, confidence, timestamp, author, lens_id, frame_id}. The lens and frame annotations are what make the contract substrate-shaped: every write knows whose view produced it and which task it served. A well-formed claim is a capability the agent gains on load. The framework grows by accretion.