Architecture
Read Path
How memory is retrieved and assembled
Read Path
The read path handles all memory retrieval, combining category summaries, vector search, and graph traversal into a coherent context.
Overview
Query
↓
Select relevant categories (LLM)
↓
Load category summaries
↓
Sufficient? → Return early
↓
Vector search for specific items
↓
Graph traversal for relationships
↓
Apply time decay ranking
↓
Assemble within token budget
↓
Return structured contextComponents
Category Selection
LLM selects which categories are relevant to the query:
async function selectCategories(
query: string,
config: DomainConfig
): Promise<string[]> {
const prompt = `
Given this query: "${query}"
Which of these memory categories are relevant?
${config.categories.map(c => `- ${c.name}: ${c.description}`).join('\n')}
Return a JSON array of category names, ordered by relevance.
Only include categories that would help answer the query.
`;
const response = await llm.complete(prompt);
return JSON.parse(response);
}Sufficiency Check
Determines if summaries alone can answer the query:
async function checkSufficiency(
query: string,
summaries: Record<string, string>
): Promise<boolean> {
const prompt = `
Query: "${query}"
Available summaries:
${Object.entries(summaries)
.map(([cat, sum]) => `## ${cat}\n${sum}`)
.join('\n\n')}
Can these summaries adequately answer the query?
Consider: Is specific detail needed? Are there gaps?
Answer: YES or NO
`;
const response = await llm.complete(prompt);
return response.trim().toUpperCase() === 'YES';
}Vector Search
Finds semantically similar items:
async function vectorSearch(
appId: string,
userId: string,
queryEmbedding: number[],
options: SearchOptions
): Promise<MemoryItem[]> {
return db.execute(sql`
SELECT * FROM search_memories_with_decay(
${appId},
${userId},
${queryEmbedding}::vector,
${options.limit || 20},
${options.decayDays || 30},
${options.minConfidence || 0.6}
)
`);
}Graph Traversal
Finds related entities:
async function traverseGraph(
appId: string,
userId: string,
entityNames: string[],
maxHops: number = 2
): Promise<GraphContext[]> {
const result = await neo4j.run(`
MATCH (u:User {app_id: $appId, id: $userId})-[:MENTIONED]->(e:Entity)
WHERE e.name IN $entityNames AND e.archived = false
MATCH path = (e)-[:RELATES_TO*1..${maxHops}]-(related:Entity)
WHERE all(r in relationships(path) WHERE r.archived = false)
AND related.archived = false
RETURN DISTINCT related,
[r in relationships(path) | r.type] as relTypes
LIMIT 20
`, { appId, userId, entityNames });
return result.records.map(formatGraphContext);
}Time Decay Ranking
Applies temporal decay to scores:
function applyTimeDecay(
items: MemoryItem[],
decayDays: number = 30
): RankedItem[] {
return items.map(item => {
const ageDays = daysSince(item.createdAt);
const decayFactor = 1 / (1 + ageDays / decayDays);
return {
...item,
finalScore: item.confidence * decayFactor * item.vectorScore,
};
}).sort((a, b) => b.finalScore - a.finalScore);
}Context Assembly
Combines all sources within token budget:
async function assembleContext(
summaries: Record<string, string>,
items: RankedItem[],
graphContext: GraphContext[],
maxTokens: number
): Promise<MemoryContext> {
let tokenCount = 0;
const result: MemoryContext = {
summaries: {},
relevantItems: [],
graphContext: [],
tokenCount: 0,
};
// 1. Add summaries (highest priority)
for (const [category, summary] of Object.entries(summaries)) {
const tokens = countTokens(summary);
if (tokenCount + tokens <= maxTokens * 0.5) {
result.summaries[category] = summary;
tokenCount += tokens;
}
}
// 2. Add relevant items
for (const item of items) {
const tokens = countTokens(item.content);
if (tokenCount + tokens <= maxTokens * 0.8) {
result.relevantItems.push(item);
tokenCount += tokens;
}
}
// 3. Add graph context
for (const ctx of graphContext) {
const tokens = countTokens(JSON.stringify(ctx));
if (tokenCount + tokens <= maxTokens) {
result.graphContext.push(ctx);
tokenCount += tokens;
}
}
result.tokenCount = tokenCount;
return result;
}Response Format
interface MemoryContext {
summaries: Record<string, string>;
relevantItems: Array<{
id: string;
content: string;
category: string;
confidence: number;
score: number;
createdAt: string;
}>;
graphContext: Array<{
name: string;
type: string;
properties: Record<string, any>;
relationships: Array<{
type: string;
target: string;
}>;
}>;
tokenCount: number;
processingTime: number;
}Performance
| Stage | Latency | Notes |
|---|---|---|
| Category selection | ~500ms | LLM call |
| Summary lookup | ~20ms | Postgres |
| Sufficiency check | ~300ms | Optional LLM |
| Vector search | ~50ms | pgvector |
| Graph traversal | ~100ms | Neo4j |
| Context assembly | ~10ms | CPU |
| Total | ~700ms | Typical |
| Worst case | ~1.5s | All stages |
Failure Modes
Neo4j Unavailable
- System continues with vector-only retrieval
- Graph context omitted from results
- Logged as warning
LLM Service Unavailable
- Falls back to direct vector search
- All categories loaded (no selection)
- Higher latency but functional