Living Memory & Alexandria

Executive Summary

We've built a shared memory system for AI agents that lets them learn from each other's sessions, share precedent across workflows, and maintain that knowledge automatically. The system is in active use on two workflows (core platform development and Salesforce development), where we're dogfooding it to prove out the architecture and fine-tune the real-world experience. The overall design has been pressure-tested against 30+ external sources from academia, industry, and thought leadership, though the current implementation doesn't yet address all of the findings.

The architecture decomposes into three layers: Memory (storage), Learning (extraction and maintenance), and Personalization (serving the right knowledge to the right agent). We operate at Level 4 (agentic learning and memory) today. Level 5 (Alexandria: the full knowledge library with propositional claims and graph traversal) is currently being built for personal and intra-team use, architected to scale.

This document focuses on the architecture: how it works, step-by-step processes, honest challenges, and the vision for what it could become. For the business case and landscape context, see the companion decks (Alexandria pitch book and Memory, Context & Graphs landscape primer).

What's Built and Running

Everything below is built and running, used daily on the PDT and SFDC dev workflows. The goal is to employ it as a real use case, iterate, and prove out the architecture before any broader rollout.

Memory & Learning Core

Harness Engineering & Governance

Alexandria & Knowledge Presentation

In Progress & Vision

Architecture: Memory / Learning / Personalization

The architecture spans three layers with explicit boundaries between them. Failures concentrate at those boundaries: conflating a retrieval problem (Personalization) with a storage problem (Memory) or an extraction problem (Learning) is the most common design mistake in agent memory systems.

flowchart TB
  subgraph P["Personalization — Real-time serving
Context engineering (Cherny)"]
    direction LR
    gc["getContext"]
    rf["Retrieval filters"]
    tb["Token budgets"]
    pd["Progressive disclosure"]
    gc --- rf --- tb --- pd
  end

  subgraph L["Learning — Async extraction
Structured artifacts (PAL, Karpathy)"]
    direction LR
    refl["Session-end reflection"]
    dp["Digestive pipeline"]
    cond["Condensation"]
    forg["Forgetting"]
    refl --- dp --- cond --- forg
  end

  subgraph M["Memory — Storage infrastructure
Vault as identity (Cornelius), File-backed state (NLAH)"]
    direction LR
    les["Lessons DB"]
    conn["Connections"]
    ent["Entities"]
    sp["Scratchpads"]
    dt["Decision traces"]
    les --- conn --- ent --- sp --- dt
  end

  P -->|"Usage signals + outcomes"| M
  M -->|"Raw material"| L
  L -->|"Structured knowledge"| P
      

The Learning Loop (Step-by-Step)

Every agent session follows the same lifecycle. The loop is the same whether the agent is working on a core platform ticket or a Salesforce development task.

sequenceDiagram
    participant Agent
    participant API as HTTP API
    participant Store as Learning Store
    participant Pipeline as Digestive Pipeline

    Agent->>API: POST /sessions (register)
    API-->>Agent: Session ID + scratchpad

    Agent->>API: GET /sessions/{id}/context
    API->>Store: Query lessons, precedent (workflow-scoped)
    Store-->>API: Relevant lessons + decision traces
    API-->>Agent: Context payload (token-budgeted)

    Note over Agent: Agent works (skills, tools, code)
    Note over Agent: Updates scratchpad every 3-5 actions

    Agent->>API: DELETE /sessions/{id} (end session)
    Note over Agent: Includes: summary, reflection,
lessons, decision traces

    API->>Store: Write lessons + traces (scoped)
    API->>Pipeline: Queue digestive processing
    Pipeline->>Store: Reduce → Reflect → Reweave → Verify → Archive
    Note over Store: Next getContext returns
these lessons
      

1. Register session. Agent identifies itself, declares what it's working on. System creates or loads a scratchpad (per-task state). Per Cornelius: the vault encounter IS identity — the agent becomes itself by reading what previous sessions left behind.
2. Load context (getContext). System retrieves relevant lessons and precedent, scoped by workflow and task similarity, within a token budget (Cherny's context engineering). Optionally includes Alexandria index and rendered claims.
3. Do the work. Agent uses skills, tools, and files. Updates scratchpad with state, decisions, and progress. Following PAL's insight: agents produce structured artifacts (code, scratchpads, decision traces), not free-form prose. Code-as-reasoning.
4. End session with reflection. Agent submits: summary of what was done, structured reflection (what was learned, what to watch for), and optionally guardrail suggestions, knowledge capture, or warnings for future sessions.
5. System writes and digests. Lessons and decision traces are stored (workflow-scoped). The digestive pipeline processes them asynchronously: extracting atomic claims, finding connections, updating existing knowledge, validating, and archiving source material.
6. Next session benefits. The next agent's getContext returns these lessons. The loop compounds. Per Karpathy's LLM OS: the knowledge store is the file system of the agent operating system; each session is a process that reads from and writes to it.

The Digestive Pipeline

Raw knowledge entering the system is not yet useful. It is food, not capability. The digestive pipeline transforms it. Each phase runs in isolation with fresh context to prevent contamination between steps.

flowchart LR
    R["Reduce
Raw → atomic claims"]
    RE["Reflect
Find connections"]
    RW["Reweave
Update existing"]
    V["Verify
Schema + dedup"]
    A["Archive
Mark processed"]
    R --> RE --> RW --> V --> A
      

How Retrieval Works

Retrieval is purpose-aware and token-budgeted. The same entity returns different claim subsets depending on the consumer's workflow and task.

flowchart TB
    Q["Agent calls getContext
(workflow, task, scopes)"]
    F["Filter by workflow + scope
(project → team → framework → org)"]
    R["Rank by: priority × recency
× tag match × value-per-token"]
    B["Apply token budget
(cap total context size)"]
    BP["Berrypicking: multi-pass
(reassess, refine, re-query)"]
    A["Assemble response:
lessons + precedent + (optional) Alexandria index"]

    Q --> F --> R --> B --> BP --> A
      

The Knowledge Graph

The graph is entity-centric: it models the real things the organization cares about and the relationships between them. Entities are the nouns of institutional knowledge.

flowchart LR
    subgraph SRC["Sources"]
        direction TB
        s1["Agent reflections"]
        s2["Research"]
        s3["Conversations"]
        s4["Events"]
    end

    subgraph INC["Input Contract"]
        c1["entity + claim +
source + confidence +
timestamp + author"]
    end

    subgraph KG["Knowledge Graph"]
        direction TB
        e["Entities"]
        r["Relationships"]
        kc["Claims"]
        b["Backlinks"]
        e --- r
        r --- kc
        kc --- b
    end

    subgraph OC["Output Contract"]
        o1["Purpose-aware
context assembly"]
    end

    subgraph CON["Consumers"]
        direction TB
        a1["Agents"]
        a2["Dashboards"]
        a3["Decision support"]
        a4["Search"]
    end

    SRC --> INC --> KG --> OC --> CON
      

Agent Taxonomy

Not every agent should learn. The architecture serves four types of agents, distinguished by their relationship to knowledge. Moving along this spectrum is incremental.

flowchart LR
    subgraph S["Stateless Tools"]
        s1["Code formatter
Data validator"]
    end
    subgraph C["Knowledge Consumers"]
        c1["Pipeline reporter
Meeting prep agent"]
    end
    subgraph L["Scoped Learners"]
        l1["Coding assistant
Sales assistant"]
    end
    subgraph F["Full-Metabolism Agents"]
        f1["Strategic architect
Business analyst"]
    end

    S -->|"Add graph reads"| C
    C -->|"Add feedback loop"| L
    L -->|"Add metabolic machinery"| F
      

The key design choice: the graph must serve stateless tools and knowledge consumers (the majority) with minimal friction. A single API call returns what they need. No session registration, no scratchpad. The metabolic machinery exists for the 1-3% that justify it.

Honest Challenges

We pressure-tested the architecture against 30+ external sources across 10+ sessions. Seven validated strengths. Five remaining gaps.

Validated strengths (protect these)

Remaining gaps (honest assessment)

Architecture Vision

The vision is federated memory with two tiers and a governed bridge. We're building this for personal and intra-team use today, architected to scale. The local tier is operational. The organizational tier is where the knowledge graph, propositional claims, and graph traversal live. Quality gates apply at the promotion boundary, not at local writes.

flowchart TB
    subgraph today["Current State (operational)"]
        L3["Living Memory store
(lessons, connections, entities)"]
        L4["Agentic learning loop
(getContext, reflection, shared)"]
        harness["Harness engineering
(evals, guardrails, failure taxonomy)"]
        L3 --> L4
        L4 --> harness
    end

    subgraph target["Target State (in progress)"]
        local["Local tier
.md files, git-backed
personal/team scope
Cornelius: vault as identity"]
        bridge["Bridge
Quality gates at
promotion boundary
Wiki gates (built)"]
        org["Alexandria graph
Propositional claims
graph traversal
Karpathy: LLM OS file system"]
        ctx["Context serving
Token-budgeted
berrypicking, disclosure
Cherny: context engineering"]
        local <-->|"query / promote"| bridge
        bridge <-->|"governed"| org
        org --> ctx
    end

    L4 -.->|"feeds"| bridge
    L3 -.->|"foundation"| org
      

The federated principle

The filesystem-vs-graph divide is not either/or. Individual developers use lightweight, file-based memory (markdown in a workspace, git-backed). Per Cornelius: "the vault is the riverbed, the sessions are the water." The graph tier provides the governed, cross-domain knowledge layer. The architecture serves both without forcing one onto the other.

The Knowledge Interchange Layer

Any source publishes knowledge into the graph through one contract: {entity, claim, source, confidence, timestamp, author}. Per Cornelius: notes aren't reminders — they're callable functions. A well-formed claim is a capability the agent gains on load. The framework grows by accretion, not revolution.