OpenLoomi Memory System

Last verified against the current repository on June 10, 2026, including the memory pipeline merged in PR #154.

This document describes how OpenLoomi stores, compacts, embeds, and retrieves memory. It is intended for maintainers working on the lifecycle engine, raw message stores, insights, RAG, vector backends, APIs, or agent tools. For a shorter product comparison, see Memory Capabilities Comparison.

OpenLoomi does not have one physical "memory database". The product-level context atlas is assembled from several isolated data sources:

• Raw memory stores original or near-original messages.
• Lifecycle summaries compact older raw memory.
• Insights store LLM-derived interpretations.
• Knowledge stores uploaded document chunks through RAG.
• Filesystem memory stores inspectable local Markdown and JSON files.
• Unified semantic search queries raw memory, insights, and knowledge, then returns one globally ranked result list.

There is no standalone context_atlas table or service. "Context atlas" is the product concept formed by these sources and their retrieval paths.

Table of Contents

• Architecture Overview
• Design Principles
• End-to-End Data Flows
• Core Memory Contracts
• Raw Message Ingestion
• Memory Tiers
• Forgetting Engine
• Retention Scoring
• Summarization
• Persistence and Index Architecture
• Embeddings and Vector Backends
• Query and Recall APIs
• MCP and Agent Recall
• Failure and Fallback Behavior
• Testing
• Maintenance Checklist
• Implementation References

Architecture Overview

The lifecycle engine in packages/ai/src/memory/ is storage-independent:

connector/chat ingestion
  -> RawMessage
  -> raw_messages
       | memoryStage: short / mid / long
       | access and importance metadata
       | optional embedding
       v
  MemoryStorageAdapter
       v
  createMemoryForgettingEngine().runCycle()
       | select old candidates
       | calculate retention score
       | group by time and dimensions
       | create MemorySummary
       | promote records between tiers
       | optionally archive details
       v
  memory_summaries

The main read paths are separate:

Lifecycle-aware query
  -> MemoryQueryApi.queryWithFallback()
  -> raw records first
  -> lifecycle summaries when raw results are insufficient

Engine-level vector recall
  -> MemoryQueryApi.semanticRecall()
  -> MemoryStorageAdapter.semanticRecallRaw()
  -> native vector search or stored-embedding cosine fallback

Application-level semantic search
  -> POST /api/memory/search
  -> searchUnifiedMemory()
       -> raw memory semantic search
       -> insight semantic search
       -> knowledge/RAG semantic search
  -> global similarity ranking

These paths share data, but they solve different problems. Summary fallback preserves lifecycle continuity. Engine recall retrieves MemoryRecords. Unified search combines multiple corpora for applications and agent tools.

The application-level unified path is semantic-only. It does not run the legacy raw-message keyword query, read lifecycle summaries, or scan filesystem memory. Exact phrase lookup and source-specific pagination remain available through their existing tools and APIs.

Design Principles

The implementation follows four boundaries that are easy to miss when reading one file at a time.

1. Memory Is a Set of Domains

Raw messages, lifecycle summaries, insights, RAG chunks, and local files are not aliases for one record type. They have different owners, metadata, update paths, and deletion semantics.

2. Canonical Data Owns Meaning

Database rows and local files are canonical records. Chroma collections and sqlite-vec tables are searchable indexes. An index can be cleared or rebuilt without redefining the source record.

3. Lifecycle and Retrieval Are Orthogonal

The forgetting engine decides how long raw details remain hot and when to create compact summaries. Semantic search decides which indexed records are similar to a query. Moving a record from short to mid does not automatically reduce its vector-search score.

4. Unification Happens at Interfaces

OpenLoomi uses shared contracts at two levels:

• UnifiedVectorSearchService presents one API over one vector backend or collection.
• searchUnifiedMemory() presents one query/result contract over raw memory, insights, and knowledge.

Neither requires all source data to live in one physical table. One giant memory bucket would be easy to draw and rather less charming to maintain.

End-to-End Data Flows

Raw Memory Write and Repair

connector or chat payload
  -> normalize to RawMessage
  -> upsert canonical raw_messages row by messageId
  -> persist lifecycle and embedding metadata
  -> embedding dream scans missing/model-changed/content-changed records
  -> update stored embedding
  -> synchronize configured sqlite-vec or Chroma index

Insight Write and Repair

insight create or update
  -> persist structured insight fields
  -> build stable embedding content and hash
  -> persist insight_embeddings record
  -> synchronize configured sqlite-vec or Chroma index
  -> scheduled dream repairs omissions or stale vectors

Lifecycle summaries and insights are intentionally different. A lifecycle summary is rule-based compaction of raw records. An insight is interpreted application data produced by the insight pipeline.

Knowledge Ingestion

document upload
  -> parse text
  -> split into overlapping chunks
  -> generate chunk embeddings
  -> persist document and chunk rows
  -> add vector chunks to sqlite-vec or Chroma in Tauri
     or use pgvector-backed search in server deployments

Cross-Source Recall

authenticated semantic query
  -> selected raw-memory branch
  -> selected insight branch
  -> selected knowledge branch
  -> normalize source-specific hits
  -> global similarity sort
  -> stable source/ID tie break
  -> top N evidence returned to API or MCP caller

The source branches currently generate their query embeddings independently. Deployments should therefore keep embedding models compatible when their scores will be compared in one global list.

Core Memory Contracts

Core contracts live in packages/ai/src/memory/contracts.ts.

MemoryRecord

MemoryRecord is the lifecycle engine representation of one raw memory item.

MemoryRecord is not a second copy of raw_messages. Storage adapters map a persisted RawMessage into this engine-level shape.

MemorySummary

MemorySummary is a rule-based compaction artifact created during tier transitions. It contains:

• the source time window,
• source and summary tiers,
• source record IDs,
• key points and keywords,
• summary text,
• grouping dimensions,
• an optional quality score.

A lifecycle summary is not an insight. Insights are LLM-derived records owned by the insights subsystem and have their own embedding/search pipeline.

MemoryStorageAdapter

MemoryStorageAdapter separates engine logic from persistence:

The main adapter is createIndexedDBMemoryStorageAdapter() in packages/indexeddb/src/forgetting.ts. Despite its historical name, it bridges the shared manager shape used by browser IndexedDB and server-selected raw message stores.

Raw Message Ingestion

Connector and chat payloads are normalized into RawMessage records. The shared storage shape is defined in packages/indexeddb/src/storage.ts.

New records normally begin with:

Raw messages are upserted by messageId. Stable IDs are important because connector and insight refresh jobs can ingest the same history repeatedly.

normalizeMemoryRecordForIngest() in packages/ai/src/memory/ingest.ts provides the equivalent engine-level normalization and defaults a missing tier to short.

Memory Tiers

Raw records use three lifecycle tiers:

Transitions create summary tiers:

The L3 type/helper exists, but the current engine loop does not run a normal long -> ... phase.

Changing a tier does not currently apply an automatic semantic-search penalty. Semantic ranking is based on vector similarity. Tiers can be supplied as filters to engine-level recall.

Forgetting Engine

createMemoryForgettingEngine() in packages/ai/src/memory/engine.ts implements progressive compaction. "Forgetting" here means cooling and summarizing before irreversible deletion.

The default policy in packages/ai/src/memory/policy.ts is:

For each user, runCycle():

1. Acquires memory_forgetting:<userId>.
2. Runs the short -> mid phase.
3. Runs the mid -> long phase.
4. Releases the lock in a finally block.

Each phase:

1. Loads source-tier records older than the phase cutoff.
2. Calculates retention scores.
3. Excludes pinned, archived, and high-retention records.
4. Groups eligible records by time bucket and configured dimensions.
5. Skips groups smaller than three records by default.
6. Creates one rule-based summary per group.
7. Persists the summary and promotes its source records.
8. Archives source details when the target tier is long and the adapter supports archival.

dryRun executes selection, scoring, grouping, and counting without writes. The bridge-level runMemoryForgettingCycle() can also perform an optional hard delete when hardDeleteArchivedOlderThan is explicitly provided. Archive and hard delete are intentionally separate operations.

Retention Scoring

DefaultMemoryRecordScorer in packages/ai/src/memory/scorer.ts returns a score in [0, 1]. A higher score means the record should remain hot longer.

score = clamp01(
  0.35 * recency +
  0.30 * access +
  0.25 * importance +
  0.10 * media +
  pinnedBoost
)

Signals:

• Recency decays linearly over 180 days.
• Access uses a logarithmic score based on accessCount.
• Importance is the maximum of explicit importance and inferred keywords.
• Media receives 0.7 when present and 0.25 otherwise.
• Pinned records receive a 0.3 boost and are also excluded from transition eligibility.

Inferred importance recognizes terms such as deadline, urgent, risk, decision, blocker, action item, milestone, bug, and incident.

Summarization

RuleBasedMemorySummarizer in packages/ai/src/memory/summarizer.ts does not call an LLM.

For each group it:

1. Sorts records chronologically.
2. Selects up to five unique text highlights.
3. Truncates highlights to 180 characters.
4. Extracts up to 12 keywords with token frequency and stop-word filtering.
5. Writes the source window, transition, record count, and highlights.
6. Assigns quality 0.75 when text highlights exist or 0.45 otherwise.

The MemorySummarizer interface is injectable, so a future summarizer can replace this implementation without changing the engine.

Persistence and Index Architecture

OpenLoomi keeps canonical records and vector indexes conceptually separate. The database row remains the source record; Chroma or a dimension-specific sqlite-vec table is a searchable index that can be repaired or rebuilt.

Browser IndexedDB

packages/indexeddb/src/manager.ts owns:

• raw_messages,
• memory_summaries,
• indexes including userId_memoryStage, userId_timestamp, userId_summaryTier, and userId_endTimestamp.

Browser IndexedDB also supports stored embeddings. When no native semantic search method exists, the memory adapter scans bounded candidates and computes cosine similarity in process.

SQLite

Tauri uses packages/sqlite/src/raw-message-manager.ts and packages/sqlite/src/schema.ts.

SQLite includes:

• raw_messages,
• memory_summaries,
• raw_messages_fts and synchronization triggers,
• dimension-specific sqlite-vec tables such as raw_messages_vec_d1024,
• delete triggers that remove vector rows when source messages are deleted.

Dimension-specific tables prevent vectors from different embedding models from being compared accidentally.

Postgres

Server deployments use apps/web/lib/memory/postgres-raw-message-store.ts and the Postgres Drizzle schema. The manager implements the same raw message, lifecycle summary, embedding update, and semantic-search operations. Raw memory and insight search use pgvector distance queries in server mode.

The server-side selector in apps/web/lib/memory/raw-message-store.ts chooses between SQLite and Postgres:

Tauri server runtime -> SQLite
web/server runtime   -> Postgres

Browser IndexedDB is a separate client/local manager path. It is adapted by the same memory contracts, but the authenticated Next.js raw-message route does not select IndexedDB as a server backend.

Files and Session State

packages/ai/src/store/conversation-store.ts stores connector conversations in per-day JSON files:

{memoryDir}/{prefix}/YYYY-MM-DD.json

In Tauri, apps/web/lib/ai/memory/chat-sync.ts exports chat Markdown under:

<appDataDir>/data/memory/chats/YYYY-MM-DD/<title>-<chatId>.md

The broader local directory also contains people, projects, notes, and strategy. These files are not automatically inserted into raw memory or RAG; they require an explicit indexing/import path and are not searched by searchUnifiedMemory().

apps/web/lib/session/context.ts manages temporary login and insight-processing state. It uses Redis when configured and ioredis-mock in Tauri/development. Its 30-minute session state is operational context, not durable memory.

Embeddings and Vector Backends

The embedding provider is shared through @openloomi/rag. It can use a cloud provider or local Transformers.js models.

Raw memory, insights, and knowledge remain isolated sources. Each source can select its own backend:

VECTOR_STORE_BACKEND=sqlite-vec

RAW_MESSAGE_VECTOR_STORE_BACKEND=sqlite-vec
INSIGHT_VECTOR_STORE_BACKEND=sqlite-vec
RAG_VECTOR_STORE_BACKEND=sqlite-vec

Supported backends in the current local implementation are:

• sqlite-vec,
• ChromaDB client-server mode.

Server deployments can use the Postgres/pgvector paths already implemented by raw memory, insights, and RAG. sqlite-vec is used by the Tauri runtime; the RAG service does not open it in a normal web/server process.

See Local Embeddings and Vector Backends for runtime configuration and table/collection details.

Two Different "Unified" Layers

Two similarly named layers solve different problems:

UnifiedVectorSearchService lives in packages/ai/rag/src/unified-vector-search-service.ts. It can initialize a Chroma, sqlite-vec, or custom IVectorStore; generate missing embeddings; upsert records; search by text or vector; filter common metadata; delete old records; and report backend statistics.

The application memory search does not put raw memory, insights, and knowledge into one collection through that class. Each corpus keeps its own persistence, metadata, retention rules, and search adapter. "Unified" at the application layer means one request and one result contract, not one giant vector table wearing three hats.

Embedding Refresh

Raw message and insight embedding maintenance regenerates vectors when:

• the vector is missing,
• the embedding model changed,
• the embedding content hash changed.

Embedding metadata is persisted with the source record and synchronized to the configured vector backend. Chroma receives explicit embeddings; OpenLoomi does not rely on Chroma's default embedding function.

The desktop scheduler runs raw-message and insight embedding repair jobs on 24-hour windows. The raw-message timestamp is tracked in process; the insight timestamp is also persisted in user insight settings. The raw-message job scans up to 100 candidates, updates stale or missing vectors, then synchronizes recent compatible vectors to Chroma when enabled. Insight write paths also generate embeddings incrementally, while the scheduled dream repairs omissions without blocking normal writes.

Query and Recall APIs

Raw Query with Summary Fallback

MemoryQueryApi.queryWithFallback():

1. Queries raw records.
2. If raw results are below minRawResultsWithoutFallback, queries lifecycle summaries for the remaining capacity.
3. Merges raw and summary hits by descending timestamp.
4. Marks returned raw records as accessed.

The web route exposes this through /api/memory/raw-messages with action query and includeSummaryFallback: true.

Engine-Level Semantic Recall

MemoryQueryApi.semanticRecall() accepts:

• queryEmbedding,
• limit and threshold,
• optional tiers,
• optional time range,
• optional dimensions.

It calls MemoryStorageAdapter.semanticRecallRaw(), wraps hits with sourceType: "raw", and marks returned records as accessed.

The IndexedDB bridge:

1. Uses manager.searchMessagesSemantically() when the active manager provides native semantic search.
2. Over-fetches before applying engine-only tier and arbitrary dimension filters.
3. Otherwise scans bounded raw candidates and computes cosine similarity over stored MemoryRecord.embedding values.

This is an engine API, not a separate HTTP route.

Unified Semantic Search

POST /api/memory/search is the cross-source application endpoint:

{
  "query": "User's last project feedback",
  "sources": ["memory", "insights", "knowledge"],
  "limit": 10,
  "threshold": 0.7
}

searchUnifiedMemory() in apps/web/lib/memory/unified-search.ts:

1. Normalizes the query, source list, limit, and threshold.
2. Searches every selected source independently with the requested per-source limit.
3. Converts every hit into UnifiedMemorySearchResult.
4. Sorts the combined hits by descending similarity.
5. Uses type and ID as stable tie breakers.
6. Returns the global top N.

Source behavior:

Unified search is semantic-only. Existing source-specific MCP tools can still perform exact or keyword lookup when literal matching is required.

The current implementation asks each selected source branch to generate its own query embedding. It does not yet calculate one vector and share it across all three branches. In practice, deployments should keep the corpora on compatible embedding models when their similarity scores will be compared globally. Cross-model similarity scores are not automatically calibrated.

Important Chroma behavior: if a Chroma query succeeds with zero results, raw memory search does not also query the database backend. Database fallback is used when Chroma is disabled or throws.

The endpoint returns:

interface UnifiedMemorySearchResult {
  type: "memory" | "insight" | "knowledge";
  id: string;
  content: string;
  similarity: number;
  metadata: Record<string, unknown>;
}

The three source stores remain isolated. "Unified" describes the query and result contract, not a requirement to put all vectors into one collection.

MCP and Agent Recall

The business-tools MCP server exposes the cross-source search as searchUnifiedMemory. Its localized UI label is "Searching Semantic Memory" / "语义记忆搜索".

The tool description tells the agent to use it early for broad questions about past conversations, projects, people, decisions, risks, owners, next actions, uploaded documents, or extracted insights. It defaults to all three sources unless the request clearly belongs to only one corpus.

The MCP defaults intentionally favor recall more than the HTTP endpoint:

For model-facing output, each result includes its source type, similarity, ID, selected metadata, and content. Individual content is capped at 1,200 characters to control tool-context growth. The tool also logs hit counts, maximum/average scores by source, warnings, and previews of the top five results.

Source-specific tools remain useful after semantic recall:

• raw-message tools provide keyword lookup and pagination,
• insight tools expose structured task/event fields,
• knowledge tools support document-specific workflows and full-document reads,
• filesystem-memory tools inspect local paths not covered by unified search.

This separation is deliberate: semantic recall is the broad first pass; exact or structured tools are the microscope.

HTTP and Raw Memory APIs

POST /api/memory/raw-messages supports:

POST /api/memory/search is the authenticated cross-source semantic endpoint. The semantic memory MCP tool calls the same unified search service. Legacy source-specific tools remain available for narrower exact/keyword workflows.

All web routes replace caller-provided ownership with the authenticated session.user.id.

Failure and Fallback Behavior

Runtime logs identify the actual semantic backend:

[UnifiedMemory] Raw message semantic search completed { backend: 'chroma', ... }
[SQLite Raw Messages] Semantic search completed { backend: 'sqlite-vec', ... }
[InsightSearch] Semantic search completed { backend: 'sqlite-vec', ... }
[RAG] Vector search completed { backend: 'sqlite-vec', ... }
[SemanticMemoryTool] search completed { hitSources, hitSourceCounts, ... }

Fallbacks use explicit stored-embedding-fallback logging instead of silently changing behavior. The Tauri RAG database fallback is a compatibility path, not true semantic ranking; maintainers should treat that log path as degraded mode.

Testing

Focused lifecycle and retrieval tests:

pnpm --filter web exec vitest run \
  tests/unit/memory-forgetting.test.ts \
  tests/unit/indexeddb-forgetting.test.ts \
  tests/unit/memory-embedding.test.ts \
  tests/unit/indexeddb-memory-embedding.test.ts \
  tests/unit/insight-embedding.test.ts \
  tests/unit/insight-search.test.ts \
  tests/unit/unified-vector-search-service.test.ts \
  tests/unit/unified-memory-search.test.ts

Backend-specific tests:

pnpm --filter web exec vitest run \
  tests/unit/sqlite-raw-message-storage.test.ts \
  tests/unit/postgres-raw-message-store.test.ts \
  tests/unit/sqlite-vec-store.test.ts \
  tests/unit/chroma-vector-store.test.ts

Maintenance Checklist

• When MemoryRecord changes, update adapter mapping and embedding text generation.
• When RawMessage changes, update IndexedDB, SQLite, Postgres, API serialization, and storage contract tests.
• When lifecycle policy or scoring changes, update forgetting tests and this document.
• When embedding text changes, bump the content-hash version so stale vectors regenerate.
• When a vector dimension changes, use dimension-specific sqlite-vec tables or a new/cleared Chroma collection.
• When unified search changes, update apps/web/tests/unit/unified-memory-search.test.ts.
• Audit every seconds/milliseconds conversion when changing timestamp handling.
• Keep lifecycle summaries, insights, and RAG chunks conceptually distinct.

Implementation References