✓ Verified 💻 Development ✓ Enhanced Data

Tiered Memory

EvoClaw Tiered Memory Architecture v2.2.0 - LLM-powered three-tier memory system with automatic dail

Rating: 4.9 (90 reviews)
Downloads: 715 downloads
Version: 1.0.0

Overview

EvoClaw Tiered Memory Architecture v2.2.0 - LLM-powered three-tier memory system with automatic daily note.

Complete Documentation

View Source →

Tiered Memory System v2.2.0

A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. 🧠

EvoClaw-compatible three-tier memory system inspired by human cognition and PageIndex tree retrieval.

What's New in v2.2.0

🔄 Automatic Daily Note Ingestion

Consolidation (daily/monthly/full modes) now auto-runs ingest-daily
Bridges memory/YYYY-MM-DD.md files → tiered memory system
No more manual ingestion required — facts flow automatically
Fixes the "two disconnected data paths" problem

What's New in v2.1.0

🎯 Structured Metadata Extraction

Automatic extraction of URLs, shell commands, and file paths from facts
Preserved during distillation and consolidation
Searchable by URL fragment

✅ Memory Completeness Validation

Check daily notes for missing URLs, commands, and next steps
Proactive warnings for incomplete information
Actionable suggestions for improvement

🔍 Enhanced Search

Search facts by URL fragment
Get all stored URLs from warm memory
Metadata-aware fact storage

🛡️ URL Preservation

URLs explicitly preserved during LLM distillation
Fallback metadata extraction if LLM misses them
Command-line support for adding metadata manually

Architecture

text

┌─────────────────────────────────────────────────────┐
│              AGENT CONTEXT (~8-15KB)                │
│                                                     │
│  ┌──────────┐  ┌────────────────────────────────┐  │
│  │  Tree    │  │  Retrieved Memory Nodes         │  │
│  │  Index   │  │  (on-demand, 1-3KB)            │  │
│  │  (~2KB)  │  │                                │  │
│  │          │  │  Fetched per conversation      │  │
│  │  Always  │  │  based on tree reasoning       │  │
│  │  loaded  │  │                                │  │
│  └────┬─────┘  └────────────────────────────────┘  │
│       │                                             │
└───────┼─────────────────────────────────────────────┘
        │
        │ LLM-powered tree search
        │
┌───────▼─────────────────────────────────────────────┐
│              MEMORY TIERS                           │
│                                                     │
│  🔴 HOT (5KB)      🟡 WARM (50KB)     🟢 COLD (∞)  │
│                                                     │
│  Core memory       Scored facts      Full archive  │
│  - Identity        - 30-day         - Turso DB     │
│  - Owner profile   - Decaying       - Queryable    │
│  - Active context  - On-device      - 10-year      │
│  - Lessons (20 max)                                │
│                                                     │
│  Always in         Retrieved via     Retrieved via │
│  context           tree search       tree search   │
└─────────────────────────────────────────────────────┘

Design Principles

From Human Memory

Consolidation — Short-term → long-term happens during consolidation cycles
Relevance Decay — Unused memories fade; accessed memories strengthen
Strategic Forgetting — Not remembering everything is a feature
Hierarchical Organization — Navigate categories, not scan linearly

From PageIndex

Vectorless Retrieval — LLM reasoning instead of embedding similarity
Tree-Structured Index — O(log n) navigation, not O(n) scan
Explainable Results — Every retrieval traces a path through categories
Reasoning-Based Search — "Why relevant?" not "how similar?"

Cloud-First (EvoClaw)

Device is replaceable — Soul lives in cloud (Turso)
Critical sync — Hot + tree sync after every conversation
Disaster recovery — Full restore in <2 minutes
Multi-device — Same agent across phone/desktop/embedded

Memory Tiers

🔴 Hot Memory (5KB max)

Purpose: Core identity and active context, always in agent's context window.

Structure:

json

{
  "identity": {
    "agent_name": "Agent",
    "owner_name": "User",
    "owner_preferred_name": "User",
    "relationship_start": "2026-01-15",
    "trust_level": 0.95
  },
  "owner_profile": {
    "personality": "technical, direct communication",
    "family": ["Sarah (wife)", "Luna (daughter, 3yo)"],
    "topics_loved": ["AI architecture", "blockchain", "system design"],
    "topics_avoid": ["small talk about weather"],
    "timezone": "Australia/Sydney",
    "work_hours": "9am-6pm"
  },
  "active_context": {
    "projects": [
      {
        "name": "EvoClaw",
        "description": "Self-evolving agent framework",
        "status": "Active - BSC integration for hackathon"
      }
    ],
    "events": [
      {"text": "Hackathon deadline Feb 15", "timestamp": 1707350400}
    ],
    "tasks": [
      {"text": "Deploy to BSC testnet", "status": "pending", "timestamp": 1707350400}
    ]
  },
  "critical_lessons": [
    {
      "text": "Always test on testnet before mainnet",
      "category": "blockchain",
      "importance": 0.9,
      "timestamp": 1707350400
    }
  ]
}

Auto-pruning:

Lessons: Max 20, removes lowest-importance when full
Events: Keeps last 10 only
Tasks: Max 10 pending
Total size: Hard limit at 5KB, progressively prunes if exceeded

Generates: MEMORY.md — auto-rebuilt from structured hot state

🟡 Warm Memory (50KB max, 30-day retention)

Purpose: Recent distilled facts with decay scoring.

Entry format:

json

{
  "id": "abc123def456",
  "text": "Decided to use zero go-ethereum deps for EvoClaw to keep binary small",
  "category": "projects/evoclaw/architecture",
  "importance": 0.8,
  "created_at": 1707350400,
  "access_count": 3,
  "score": 0.742,
  "tier": "warm"
}

Scoring:

text

score = importance × recency_decay(age) × reinforcement(access_count)

recency_decay(age) = exp(-age_days / 30)
reinforcement(access) = 1 + 0.1 × access_count

Tier classification:

score >= 0.7 → Hot (promote to hot state)
score >= 0.3 → Warm (keep)
score >= 0.05 → Cold (archive)
score < 0.05 → Frozen (delete after retention period)

Eviction triggers:

Age > 30 days AND score < 0.3
Total warm size > 50KB (evicts lowest-scored)
Manual consolidation

🟢 Cold Memory (Unlimited, Turso)

Purpose: Long-term archive, queryable but never bulk-loaded.

Schema:

sql

CREATE TABLE cold_memories (
  id TEXT PRIMARY KEY,
  agent_id TEXT NOT NULL,
  text TEXT NOT NULL,
  category TEXT NOT NULL,
  importance REAL DEFAULT 0.5,
  created_at INTEGER NOT NULL,
  access_count INTEGER DEFAULT 0
);

CREATE TABLE critical_state (
  agent_id TEXT PRIMARY KEY,
  data TEXT NOT NULL,  -- {hot_state, tree_nodes, timestamp}
  updated_at INTEGER NOT NULL
);

Retention: 10 years (configurable) Cleanup: Monthly consolidation removes frozen entries older than retention period

Tree Index

Purpose: Hierarchical category map for O(log n) retrieval.

Constraints:

Max 50 nodes
Max depth 4 levels
Max 2KB serialized
Max 10 children per node

Example:

text

Memory Tree Index
==================================================
📂 Root (warm:15, cold:234)
  📁 owner — Owner profile and preferences
     Memories: warm=5, cold=89
  📁 projects — Active projects
     Memories: warm=8, cold=67
    📁 projects/evoclaw — EvoClaw framework
       Memories: warm=6, cold=45
      📁 projects/evoclaw/bsc — BSC integration
         Memories: warm=3, cold=12
  📁 technical — Technical setup and config
     Memories: warm=2, cold=34
  📁 lessons — Learned lessons and rules
     Memories: warm=0, cold=44

Nodes: 7/50
Size: 1842 / 2048 bytes

Operations:

--add PATH DESC — Add category node
--remove PATH — Remove node (only if no data)
--prune — Remove dead nodes (no activity in 60+ days)
--show — Pretty-print tree

Distillation Engine

Purpose: Three-stage compression of conversations.

Pipeline:

text

Raw conversation (500B)
  ↓ Stage 1→2: Extract structured info
Distilled fact (80B)
  ↓ Stage 2→3: Generate one-line summary
Core summary (20B)

Stage 1→2: Raw → Distilled

Input: Raw conversation text Output: Structured JSON

json

{
  "fact": "User decided to use raw JSON-RPC for BSC to avoid go-ethereum dependency",
  "emotion": "determined",
  "people": ["User"],
  "topics": ["blockchain", "architecture", "dependencies"],
  "actions": ["decided to use raw JSON-RPC", "avoid go-ethereum"],
  "outcome": "positive"
}

Modes:

rule: Regex/heuristic extraction (fast, no LLM)
llm: LLM-powered extraction (accurate, requires endpoint)

Usage:

bash

# Rule-based (default)
distiller.py --text "Had a productive chat about the BSC integration..." --mode rule

# LLM-powered
distiller.py --text "..." --mode llm --llm-endpoint http://localhost:8080/complete

# With core summary
distiller.py --text "..." --mode rule --core-summary

Stage 2→3: Distilled → Core Summary

Purpose: One-line summary for tree index

Example:

text

Distilled: {
  "fact": "User decided raw JSON-RPC for BSC, no go-ethereum",
  "outcome": "positive"
}

Core summary: "BSC integration: raw JSON-RPC (no deps)"

Target: <30 bytes

LLM-Powered Tree Search

Purpose: Semantic search through tree structure using LLM reasoning.

How it works:

Build prompt with tree structure + query
LLM reasons about which categories are relevant
Returns category paths with relevance scores
Fetches memories from those categories

Example:

Query: "What did we decide about the hackathon deadline?"

Keyword search returns:

projects/evoclaw (0.8)
technical/deployment (0.4)

LLM search reasons:

projects/evoclaw/bsc (0.95) — "BSC integration for hackathon"
active_context/events (0.85) — "Deadline mentioned here"

LLM prompt template:

text

You are a memory retrieval system. Given a memory tree index and a query, 
identify which categories are relevant.

Memory Tree Index:
  projects/evoclaw — EvoClaw framework (warm:6, cold:45)
  projects/evoclaw/bsc — BSC integration (warm:3, cold:12)
  ...

User Query: What did we decide about the hackathon deadline?

Output (JSON):
[
  {"path": "projects/evoclaw/bsc", "relevance": 0.95, "reason": "BSC work for hackathon"},
  {"path": "active_context/events", "relevance": 0.85, "reason": "deadline tracking"}
]

Usage:

bash

# Keyword search (fast)
tree_search.py --query "BSC integration" --tree-file memory-tree.json --mode keyword

# LLM search (accurate)
tree_search.py --query "what did we decide about hackathon?" \
  --tree-file memory-tree.json --mode llm --llm-endpoint http://localhost:8080/complete

# Generate prompt for external LLM
tree_search.py --query "..." --tree-file memory-tree.json \
  --mode llm --llm-prompt-file prompt.txt

Multi-Agent Support

Agent ID scoping — All operations support --agent-id flag.

File layout:

text

memory/
  default/
    warm-memory.json
    memory-tree.json
    hot-memory-state.json
    metrics.json
  agent-2/
    warm-memory.json
    memory-tree.json
    ...
MEMORY.md              # default agent
MEMORY-agent-2.md      # agent-2

Cold storage: Agent-scoped queries via agent_id column

Usage:

bash

# Store for agent-2
memory_cli.py store --text "..." --category "..." --agent-id agent-2

# Retrieve for agent-2
memory_cli.py retrieve --query "..." --agent-id agent-2

# Consolidate agent-2
memory_cli.py consolidate --mode daily --agent-id agent-2

Consolidation Modes

Purpose: Periodic memory maintenance and optimization.

Quick (hourly)

Warm eviction (score-based)
Archive expired to cold
Recalculate all scores
Rebuild MEMORY.md

Daily

Everything in Quick
Tree prune (remove dead nodes, 60+ days no activity)

Monthly

Everything in Daily
Tree rebuild (LLM-powered restructuring, future)
Cold cleanup (delete frozen entries older than retention)

Full

Everything in Monthly
Full recalculation of all scores
Deep tree analysis
Generate consolidation report

Usage:

bash

# Quick consolidation (default)
memory_cli.py consolidate

# Daily (run via cron)
memory_cli.py consolidate --mode daily

# Monthly (run via cron)
memory_cli.py consolidate --mode monthly --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Recommended schedule:

Quick: Every 2-4 hours (heartbeat)
Daily: Midnight via cron
Monthly: 1st of month via cron

Critical Sync (Cloud-First)

Purpose: Cloud backup of hot state + tree after every conversation.

What syncs:

Hot memory state (identity, owner profile, active context, lessons)
Tree index (structure + counts)
Timestamp

Recovery: If device lost, restore from cloud in <2 minutes

Usage:

bash

# Manual critical sync
memory_cli.py sync-critical --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN" --agent-id default

# Automatic: Call after every important conversation
# In agent code:
#   1. Process conversation
#   2. Store distilled facts
#   3. Call sync-critical

Retry strategy: Exponential backoff if cloud unreachable (5s, 10s, 20s, 40s)

Metrics & Observability

Tracked metrics:

json

{
  "tree_index_size_bytes": 1842,
  "tree_node_count": 37,
  "hot_memory_size_bytes": 4200,
  "warm_memory_count": 145,
  "warm_memory_size_kb": 38.2,
  "retrieval_count": 234,
  "evictions_today": 12,
  "reinforcements_today": 67,
  "consolidation_count": 8,
  "last_consolidation": 1707350400,
  "context_tokens_saved": 47800,
  "timestamp": "2026-02-10T14:30:00"
}

Usage:

bash

memory_cli.py metrics --agent-id default

Key metrics:

context_tokens_saved — Estimated tokens saved vs. flat MEMORY.md
retrieval_count — How often memories are accessed
evictions_today — Memory pressure indicator
warm_memory_size_kb — Storage usage

Commands Reference

Store

bash

memory_cli.py store --text "Fact text" --category "path/to/category" [--importance 0.8] [--agent-id default]

Importance guide:

0.9-1.0 — Critical decisions, credentials, core identity
0.7-0.8 — Project decisions, architecture, preferences
0.5-0.6 — General facts, daily events
0.3-0.4 — Casual mentions, low priority

Example:

bash

memory_cli.py store \
  --text "Decided to deploy EvoClaw on BSC testnet before mainnet" \
  --category "projects/evoclaw/deployment" \
  --importance 0.85 \
  --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

# Store with explicit metadata (v2.1.0+)
memory_cli.py store \
  --text "Z-Image ComfyUI model for photorealistic images" \
  --category "tools/image-generation" \
  --importance 0.8 \
  --url "https://docs.comfy.org/tutorials/image/z-image/z-image" \
  --command "huggingface-cli download Tongyi-MAI/Z-Image" \
  --path "/home/user/models/"

Validate (v2.1.0)

bash

memory_cli.py validate [--file PATH] [--agent-id default]

Purpose: Check daily notes for incomplete information (missing URLs, commands, next steps).

Example:

bash

# Validate today's daily notes
memory_cli.py validate

# Validate specific file
memory_cli.py validate --file memory/2026-02-13.md

Output:

json

{
  "status": "warning",
  "warnings_count": 2,
  "warnings": [
    "Tool 'Z-Image' mentioned without URL/documentation link",
    "Action 'install' mentioned without command example"
  ],
  "suggestions": [
    "Add URLs for mentioned tools/services",
    "Include command examples for setup/installation steps",
    "Document next steps after decisions"
  ]
}

Extract Metadata (v2.1.0)

bash

memory_cli.py extract-metadata --file PATH

Purpose: Extract structured metadata (URLs, commands, paths) from a file.

Example:

bash

memory_cli.py extract-metadata --file memory/2026-02-13.md

Output:

json

{
  "file": "memory/2026-02-13.md",
  "metadata": {
    "urls": [
      "https://docs.comfy.org/tutorials/image/z-image/z-image",
      "https://github.com/Lightricks/LTX-Video"
    ],
    "commands": [
      "huggingface-cli download Tongyi-MAI/Z-Image",
      "git clone https://github.com/Lightricks/LTX-Video.git"
    ],
    "paths": [
      "/home/peter/ai-stack/comfyui/models",
      "./configs/ltx-video-2-config.yaml"
    ]
  },
  "summary": {
    "urls_count": 2,
    "commands_count": 2,
    "paths_count": 2
  }
}

Search by URL (v2.1.0)

bash

memory_cli.py search-url --url FRAGMENT [--limit 5] [--agent-id default]

Purpose: Search facts by URL fragment.

Example:

bash

# Find all facts with comfy.org URLs
memory_cli.py search-url --url "comfy.org"

# Find GitHub repos
memory_cli.py search-url --url "github.com" --limit 10

Output:

json

{
  "query": "comfy.org",
  "results_count": 1,
  "results": [
    {
      "id": "abc123",
      "text": "Z-Image ComfyUI model for photorealistic images",
      "category": "tools/image-generation",
      "metadata": {
        "urls": ["https://docs.comfy.org/tutorials/image/z-image/z-image"],
        "commands": ["huggingface-cli download Tongyi-MAI/Z-Image"],
        "paths": []
      }
    }
  ]
}

Retrieve

bash

memory_cli.py retrieve --query "search query" [--limit 5] [--llm] [--llm-endpoint URL] [--agent-id default]

Modes:

Default: Keyword-based tree + warm + cold search
--llm: LLM-powered semantic tree search

Example:

bash

# Keyword search
memory_cli.py retrieve --query "BSC deployment decision" --limit 5

# LLM search (more accurate)
memory_cli.py retrieve \
  --query "what did we decide about blockchain integration?" \
  --llm --llm-endpoint http://localhost:8080/complete \
  --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Distill

bash

memory_cli.py distill --text "raw conversation" [--llm] [--llm-endpoint URL]

Example:

bash

# Rule-based distillation
memory_cli.py distill --text "User: Let's deploy to testnet first. Agent: Good idea, safer that way."

# LLM distillation
memory_cli.py distill \
  --text "Long conversation with nuance..." \
  --llm --llm-endpoint http://localhost:8080/complete

Output:

json

{
  "distilled": {
    "fact": "Decided to deploy to testnet before mainnet",
    "emotion": "cautious",
    "people": [],
    "topics": ["deployment", "testnet", "safety"],
    "actions": ["deploy to testnet"],
    "outcome": "positive"
  },
  "mode": "rule",
  "original_size": 87,
  "distilled_size": 156
}

Hot Memory

bash

# Update hot state
memory_cli.py hot --update KEY JSON [--agent-id default]

# Rebuild MEMORY.md
memory_cli.py hot --rebuild [--agent-id default]

# Show current hot state
memory_cli.py hot [--agent-id default]

Keys:

identity — Agent/owner identity info
owner_profile — Owner preferences, personality
lesson — Add critical lesson
event — Add event to active context
task — Add task to active context
project — Add/update project

Examples:

bash

# Update owner profile
memory_cli.py hot --update owner_profile '{"timezone": "Australia/Sydney", "work_hours": "9am-6pm"}'

# Add lesson
memory_cli.py hot --update lesson '{"text": "Always test on testnet first", "category": "blockchain", "importance": 0.9}'

# Add project
memory_cli.py hot --update project '{"name": "EvoClaw", "status": "Active", "description": "Self-evolving agent framework"}'

# Rebuild MEMORY.md
memory_cli.py hot --rebuild

Tree

bash

# Show tree
memory_cli.py tree --show [--agent-id default]

# Add node
memory_cli.py tree --add "path/to/category" "Description" [--agent-id default]

# Remove node
memory_cli.py tree --remove "path/to/category" [--agent-id default]

# Prune dead nodes
memory_cli.py tree --prune [--agent-id default]

Examples:

bash

# Add category
memory_cli.py tree --add "projects/evoclaw/bsc" "BSC blockchain integration"

# Remove empty category
memory_cli.py tree --remove "old/unused/path"

# Prune dead nodes (60+ days no activity)
memory_cli.py tree --prune

Cold Storage

bash

# Initialize Turso tables
memory_cli.py cold --init --db-url URL --auth-token TOKEN

# Query cold storage
memory_cli.py cold --query "search term" [--limit 10] [--agent-id default] --db-url URL --auth-token TOKEN

Examples:

bash

# Init tables (once)
memory_cli.py cold --init --db-url "https://your-db.turso.io" --auth-token "your-token"

# Query cold archive
memory_cli.py cold --query "blockchain decision" --limit 10 --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Configuration

File: config.json (optional, uses defaults if not present)

json

{
  "agent_id": "default",
  "hot": {
    "max_bytes": 5120,
    "max_lessons": 20,
    "max_events": 10,
    "max_tasks": 10
  },
  "warm": {
    "max_kb": 50,
    "retention_days": 30,
    "eviction_threshold": 0.3
  },
  "cold": {
    "backend": "turso",
    "retention_years": 10
  },
  "scoring": {
    "half_life_days": 30,
    "reinforcement_boost": 0.1
  },
  "tree": {
    "max_nodes": 50,
    "max_depth": 4,
    "max_size_bytes": 2048
  },
  "distillation": {
    "aggression": 0.7,
    "max_distilled_bytes": 100,
    "mode": "rule"
  },
  "consolidation": {
    "warm_eviction": "hourly",
    "tree_prune": "daily",
    "tree_rebuild": "monthly"
  }
}

Integration with OpenClaw Agents

After Conversation

python

import subprocess
import json

def process_conversation(user_message, agent_response, category="conversations"):
    # 1. Distill conversation
    text = f"User: {user_message}\nAgent: {agent_response}"
    result = subprocess.run(
        ["python3", "skills/tiered-memory/scripts/memory_cli.py", "distill", "--text", text],
        capture_output=True, text=True
    )
    distilled = json.loads(result.stdout)
    
    # 2. Determine importance
    importance = 0.7 if "decision" in distilled["distilled"]["outcome"] else 0.5
    
    # 3. Store
    subprocess.run([
        "python3", "skills/tiered-memory/scripts/memory_cli.py", "store",
        "--text", distilled["distilled"]["fact"],
        "--category", category,
        "--importance", str(importance),
        "--db-url", os.getenv("TURSO_URL"),
        "--auth-token", os.getenv("TURSO_TOKEN")
    ])
    
    # 4. Critical sync
    subprocess.run([
        "python3", "skills/tiered-memory/scripts/memory_cli.py", "sync-critical",
        "--db-url", os.getenv("TURSO_URL"),
        "--auth-token", os.getenv("TURSO_TOKEN")
    ])

Before Responding (Retrieval)

python

def get_relevant_context(query):
    result = subprocess.run(
        [
            "python3", "skills/tiered-memory/scripts/memory_cli.py", "retrieve",
            "--query", query,
            "--limit", "5",
            "--llm",
            "--llm-endpoint", "http://localhost:8080/complete",
            "--db-url", os.getenv("TURSO_URL"),
            "--auth-token", os.getenv("TURSO_TOKEN")
        ],
        capture_output=True, text=True
    )
    
    memories = json.loads(result.stdout)
    return "\n".join([f"- {m['text']}" for m in memories])

Heartbeat Consolidation

python

import schedule

# Hourly quick consolidation
schedule.every(2).hours.do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "quick",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

# Daily tree prune
schedule.every().day.at("00:00").do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "daily",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

# Monthly full consolidation
schedule.every().month.do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "monthly",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

LLM Integration

Model Recommendations

For Distillation & Tree Search:

Claude 3 Haiku (fast, cheap, excellent structure)
GPT-4o-mini (good balance)
Gemini 1.5 Flash (very fast)

For Tree Rebuilding:

Claude 3.5 Sonnet (better reasoning)
GPT-4o (strong planning)

Cost Optimization

Use cheaper models for frequent operations (distill, search)
Batch distillation — Queue conversations, distill in batch
Cache tree prompts — Tree structure doesn't change often
Skip LLM for simple — Use rule-based for short conversations

Example LLM Endpoint

python

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/complete", methods=["POST"])
def complete():
    data = request.json
    prompt = data["prompt"]
    
    # Call your LLM (OpenAI, Anthropic, local model, etc.)
    response = llm_client.complete(prompt)
    
    return jsonify({"text": response})

if __name__ == "__main__":
    app.run(port=8080)

Performance Characteristics

Context Size:

Hot: ~5KB (always loaded)
Tree: ~2KB (always loaded)
Retrieved: ~1-3KB per query
Total: ~8-15KB (constant, regardless of agent age)

Retrieval Speed:

Keyword: 10-20ms
LLM tree search: 300-600ms
Cold query: 50-100ms

5-Year Scenario:

Hot: Still 5KB (living document)
Warm: Last 30 days (~50KB)
Cold: ~50MB in Turso (compressed distilled facts)
Tree: Still 2KB (different nodes, same size)
Context per session: Same as day 1

Comparison with Alternatives

System	Memory Model	Scaling	Accuracy	Cost
Flat MEMORY.md	Linear text	❌ Months	⚠️ Degrades	❌ Linear
Vector RAG	Embeddings	✅ Years	⚠️ Similarity≠relevance	⚠️ Moderate
EvoClaw Tiered	Tree + tiers	✅ Decades	✅ Reasoning-based	✅ Fixed

Why tree > vectors:

Accuracy: 98%+ vs. 70-80% (PageIndex benchmark)
Explainable: "Projects → EvoClaw → BSC" vs. "cosine 0.73"
Multi-hop: Natural vs. poor
False positives: Low vs. high

Troubleshooting

Tree size exceeding limit

bash

# Prune dead nodes
memory_cli.py tree --prune

# Check which nodes are largest
memory_cli.py tree --show | grep "Memories:"

# Manually remove unused categories
memory_cli.py tree --remove "unused/category"

Warm memory filling up

bash

# Run consolidation
memory_cli.py consolidate --mode daily --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

# Check stats
memory_cli.py metrics

# Lower eviction threshold (keeps less in warm)
# Edit config.json: "eviction_threshold": 0.4

Hot memory exceeding 5KB

bash

# Hot auto-prunes, but check structure
memory_cli.py hot

# Remove old projects/tasks manually
memory_cli.py hot --update project '{"name": "OldProject", "status": "Completed"}'

# Rebuild to force pruning
memory_cli.py hot --rebuild

LLM search failing

bash

# Fallback to keyword search (automatic)
memory_cli.py retrieve --query "..." --limit 5

# Test LLM endpoint
curl -X POST http://localhost:8080/complete -d '{"prompt": "test"}'

# Generate prompt for external testing
tree_search.py --query "..." --tree-file memory/memory-tree.json --mode llm --llm-prompt-file test.txt

Migration from v1.x

Backward compatible: Existing warm-memory.json and memory-tree.json files work as-is.

New files:

config.json (optional, uses defaults)
hot-memory-state.json (auto-created)
metrics.json (auto-created)

Steps:

Update skill: clawhub update tiered-memory
Run consolidation to rebuild hot state: memory_cli.py consolidate
Initialize cold storage (optional): memory_cli.py cold --init --db-url ... --auth-token ...
Configure agent to use new commands (see Integration section)

Migration from v2.0 to v2.1

Fully backward compatible: Existing memory files work without changes.

What's new:

✅ Metadata automatically extracted from existing facts when loaded
✅ New commands: validate, extract-metadata, search-url
✅ store command now accepts --url, --command, --path flags
✅ Distillation preserves URLs and technical details
✅ No action required - just update and use new features

Testing the upgrade:

bash

# Update skill
clawhub update tiered-memory

# Test metadata extraction
memory_cli.py extract-metadata --file memory/2026-02-13.md

# Validate your recent notes
memory_cli.py validate

# Search by URL
memory_cli.py search-url --url "github.com"

References

Design: /docs/TIERED-MEMORY.md (EvoClaw)
Cloud Sync: /docs/CLOUD-SYNC.md (EvoClaw)
Inspiration: PageIndex (tree-based retrieval)

v2.1.0 — A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. Now with structured metadata to remember HOW, not just WHAT. 🧠🌲🔗

Installation

Terminal bash


openclaw install tiered-memory

Copied!

💻Code Examples

└─────────────────────────────────────────────────────┘

.txt

## Design Principles

### From Human Memory
- **Consolidation** — Short-term → long-term happens during consolidation cycles
- **Relevance Decay** — Unused memories fade; accessed memories strengthen
- **Strategic Forgetting** — Not remembering everything is a feature
- **Hierarchical Organization** — Navigate categories, not scan linearly

### From PageIndex
- **Vectorless Retrieval** — LLM reasoning instead of embedding similarity
- **Tree-Structured Index** — O(log n) navigation, not O(n) scan
- **Explainable Results** — Every retrieval traces a path through categories
- **Reasoning-Based Search** — "Why relevant?" not "how similar?"

### Cloud-First (EvoClaw)
- **Device is replaceable** — Soul lives in cloud (Turso)
- **Critical sync** — Hot + tree sync after every conversation
- **Disaster recovery** — Full restore in <2 minutes
- **Multi-device** — Same agent across phone/desktop/embedded

## Memory Tiers

### 🔴 Hot Memory (5KB max)

**Purpose:** Core identity and active context, always in agent's context window.

**Structure:**

}

.txt

**Auto-pruning:**
- Lessons: Max 20, removes lowest-importance when full
- Events: Keeps last 10 only
- Tasks: Max 10 pending
- Total size: Hard limit at 5KB, progressively prunes if exceeded

**Generates:** `MEMORY.md` — auto-rebuilt from structured hot state

### 🟡 Warm Memory (50KB max, 30-day retention)

**Purpose:** Recent distilled facts with decay scoring.

**Entry format:**

reinforcement(access) = 1 + 0.1 × access_count

reinforcementaccess--1--01--accesscount.txt

**Tier classification:**
- `score >= 0.7` → Hot (promote to hot state)
- `score >= 0.3` → Warm (keep)
- `score >= 0.05` → Cold (archive)
- `score < 0.05` → Frozen (delete after retention period)

**Eviction triggers:**
1. Age > 30 days AND score < 0.3
2. Total warm size > 50KB (evicts lowest-scored)
3. Manual consolidation

### 🟢 Cold Memory (Unlimited, Turso)

**Purpose:** Long-term archive, queryable but never bulk-loaded.

**Schema:**

);

.txt

**Retention:** 10 years (configurable)
**Cleanup:** Monthly consolidation removes frozen entries older than retention period

## Tree Index

**Purpose:** Hierarchical category map for O(log n) retrieval.

**Constraints:**
- Max 50 nodes
- Max depth 4 levels
- Max 2KB serialized
- Max 10 children per node

**Example:**

Size: 1842 / 2048 bytes

size-1842--2048-bytes.txt

**Operations:**
- `--add PATH DESC` — Add category node
- `--remove PATH` — Remove node (only if no data)
- `--prune` — Remove dead nodes (no activity in 60+ days)
- `--show` — Pretty-print tree

## Distillation Engine

**Purpose:** Three-stage compression of conversations.

**Pipeline:**

Core summary (20B)

core-summary-20b.txt

### Stage 1→2: Raw → Distilled

**Input:** Raw conversation text
**Output:** Structured JSON

}

.txt

**Modes:**
- `rule`: Regex/heuristic extraction (fast, no LLM)
- `llm`: LLM-powered extraction (accurate, requires endpoint)

**Usage:**

distiller.py --text "..." --mode rule --core-summary

distillerpy---text----mode-rule---core-summary.txt

### Stage 2→3: Distilled → Core Summary

**Purpose:** One-line summary for tree index

**Example:**

Core summary: "BSC integration: raw JSON-RPC (no deps)"

core-summary-bsc-integration-raw-json-rpc-no-deps.txt

**Target:** <30 bytes

## LLM-Powered Tree Search

**Purpose:** Semantic search through tree structure using LLM reasoning.

**How it works:**

1. **Build prompt** with tree structure + query
2. **LLM reasons** about which categories are relevant
3. **Returns** category paths with relevance scores
4. **Fetches** memories from those categories

**Example:**

Query: *"What did we decide about the hackathon deadline?"*

**Keyword search** returns:
- `projects/evoclaw` (0.8)
- `technical/deployment` (0.4)

**LLM search** reasons:
- `projects/evoclaw/bsc` (0.95) — "BSC integration for hackathon"
- `active_context/events` (0.85) — "Deadline mentioned here"

**LLM prompt template:**

--mode llm --llm-prompt-file prompt.txt

---mode-llm---llm-prompt-file-prompttxt.txt

## Multi-Agent Support

**Agent ID scoping** — All operations support `--agent-id` flag.

**File layout:**

Related Skills

✓ Verified 💻 Development

4claw

4claw — a moderated imageboard for AI agents.

🧠 Claude-Ready #ai_and-llms

✓ Verified 💻 Development

Aap Passport

Agent Attestation Protocol - The Reverse Turing Test.

🧠 Claude-Ready #ai_and-llms

✓ Verified 💻 Development

Acestep Lyrics Transcription

Transcribe audio to timestamped lyrics using OpenAI Whisper or ElevenLabs Scribe API.

⚡ GPT-Optimized #ai_and-llms #api #script

✓ Verified 💻 Development

Adaptive Suite

A continuously adaptive skill suite that empowers Clawdbot.

🧠 Claude-Ready #ai_and-llms #bot

Overview

Complete Documentation

Tiered Memory System v2.2.0

What's New in v2.2.0

What's New in v2.1.0

Architecture

Design Principles

From Human Memory

From PageIndex

Cloud-First (EvoClaw)

Memory Tiers

🔴 Hot Memory (5KB max)

🟡 Warm Memory (50KB max, 30-day retention)

🟢 Cold Memory (Unlimited, Turso)

Tree Index

Distillation Engine

Stage 1→2: Raw → Distilled

Stage 2→3: Distilled → Core Summary

LLM-Powered Tree Search

Multi-Agent Support

Consolidation Modes

Quick (hourly)

Daily

Monthly

Full

Critical Sync (Cloud-First)

Metrics & Observability

Commands Reference

Store

Validate (v2.1.0)

Extract Metadata (v2.1.0)

Search by URL (v2.1.0)

Retrieve

Distill

Hot Memory

Tree

Cold Storage

Configuration

Integration with OpenClaw Agents

After Conversation

Before Responding (Retrieval)

Heartbeat Consolidation

LLM Integration

Model Recommendations

Cost Optimization

Example LLM Endpoint

Performance Characteristics

Comparison with Alternatives

Troubleshooting

Tree size exceeding limit

Warm memory filling up

Hot memory exceeding 5KB

LLM search failing

Migration from v1.x

Migration from v2.0 to v2.1

References

Installation

💻Code Examples

└─────────────────────────────────────────────────────┘

}

reinforcement(access) = 1 + 0.1 × access_count

);

Size: 1842 / 2048 bytes

Core summary (20B)

}

distiller.py --text "..." --mode rule --core-summary

Core summary: "BSC integration: raw JSON-RPC (no deps)"

--mode llm --llm-prompt-file prompt.txt

Tags

Quick Info

Ready to Install?

Resources

Related Skills

4claw

Aap Passport

Acestep Lyrics Transcription

Adaptive Suite