Skip to content

Project Structure

The AutoMem server repository follows a hybrid Python/Node.js structure with three primary deployment targets: the core Flask API service, the MCP bridge server, and operational scripts.

graph TB
    Root["automem/\nRepository Root"]

    subgraph "Python Source"
        AppPy["app.py\nMain Flask application"]
        ConsolPy["consolidation.py\nBackground workers"]

        Root --> AppPy
        Root --> ConsolPy
    end

    subgraph "Node.js Integration"
        MCPDir["mcp-sse-server/\nMCP bridge service"]
        MCPNode["node_modules/\n(ignored)"]

        Root --> MCPDir
        MCPDir --> MCPNode
    end

    subgraph "Scripts & Tools"
        ScriptsDir["scripts/\nOperational scripts"]
        BackupScript["backup_automem.py"]
        RecoverScript["recover_from_qdrant.py"]
        MonitorScript["health_monitor.py"]

        Root --> ScriptsDir
        ScriptsDir --> BackupScript
        ScriptsDir --> RecoverScript
        ScriptsDir --> MonitorScript
    end

    subgraph "Configuration"
        Env[".env\n(ignored, contains secrets)"]
        Requirements["requirements.txt\nrequirements-dev.txt"]
        Dockerfile["Dockerfile"]
        RailwayToml["railway.toml"]

        Root --> Requirements
        Root --> Dockerfile
        Root --> RailwayToml
    end

app.py — Flask Orchestration Entry Point

Section titled “app.py — Flask Orchestration Entry Point”

A ~506-line orchestration file that wires together all components from the automem/ package. It is no longer a monolithic application — all business logic, routes, workers, and configuration live in the package.

ResponsibilityImplementation
Service startup and wiringImports from automem/runtime_wiring.py
Flask app factoryDelegates to automem/api/ route modules
Background worker launchDelegates to worker modules in automem/
Configuration loadingautomem/config.py

The main package containing all business logic, organized by domain:

automem/
├── __init__.py
├── config.py # All configuration constants
├── service_state.py # ServiceState dataclass
├── service_runtime.py # Runtime initialization helpers
├── service_runtime_bindings.py
├── runtime_wiring.py # Server startup orchestration
├── runtime_environment.py
├── app_helper_bindings.py
├── api/
│ ├── memory.py # POST/GET/PATCH/DELETE /memory routes
│ ├── recall.py # GET /recall routes
│ ├── graph.py # Graph query routes
│ ├── admin.py # Admin operations
│ ├── health.py # Health endpoint
│ ├── enrichment.py # Enrichment status/reprocess
│ ├── consolidation.py # Consolidation trigger/status
│ ├── viewer.py # Graph viewer routes
│ ├── stream.py # SSE streaming
│ ├── auth_helpers.py # Authentication middleware
│ ├── runtime_bootstrap.py
│ ├── runtime_memory_routes.py
│ └── runtime_recall_routes.py
├── classification/
│ ├── __init__.py
│ └── memory_classifier.py
├── consolidation/
│ ├── runtime_bindings.py
│ ├── runtime_helpers.py
│ ├── runtime_routes.py
│ └── runtime_scheduler.py
├── embedding/
│ ├── provider.py # Base provider class
│ ├── provider_init.py # Auto-selection logic
│ ├── openai.py
│ ├── voyage.py
│ ├── fastembed.py
│ ├── ollama.py
│ ├── placeholder.py
│ ├── runtime_pipeline.py
│ ├── runtime_bindings.py
│ └── runtime_helpers.py
├── enrichment/
│ ├── runtime_bindings.py
│ ├── runtime_helpers.py
│ ├── runtime_orchestration.py
│ ├── runtime_queue_bindings.py
│ └── runtime_worker.py
├── search/
│ ├── runtime_recall_helpers.py
│ ├── runtime_relations.py
│ └── runtime_keywords.py
├── stores/
│ ├── graph_store.py
│ ├── vector_store.py
│ └── runtime_clients.py
├── sync/
│ ├── runtime_bindings.py
│ └── runtime_worker.py
├── analytics/
│ └── runtime_helpers.py
└── utils/
├── entity_extraction.py
├── scoring.py
├── tags.py
├── time.py
├── text.py
├── graph.py
└── validation.py

consolidation.py — Dream-Inspired Maintenance

Section titled “consolidation.py — Dream-Inspired Maintenance”

Self-contained consolidation engine with no Flask dependencies. Designed for both scheduled background execution and manual CLI invocation.

ClassPurposeKey Methods
MemoryConsolidatorExecute consolidation tasksconsolidate(), calculate_relevance_score(), discover_creative_associations(), cluster_similar_memories(), apply_controlled_forgetting()
ConsolidationSchedulerManage task intervalsrun_scheduled_tasks(), should_run(), get_next_runs()
GraphLike (Protocol)Database abstractionquery() method only
VectorStoreProtocol (Protocol)Vector store abstractiondelete() method only

Caching optimization: Relationship count queries use @lru_cache with hourly invalidation (consolidation.py:152-176).

Translates Model Context Protocol (MCP) tool calls into AutoMem REST API requests. Enables ChatGPT, ElevenLabs, and other SSE-compatible AI platforms to access AutoMem.

  • Port: 8080 (configurable via PORT environment variable)
  • Connects to Flask API at AUTOMEM_URL (default: http://localhost:8001)
  • Key files: index.js (SSE endpoint handlers, JSON-RPC to REST translation), package.json

Production-ready Python scripts for data protection and disaster recovery.

ScriptPurposeDependencies
backup_automem.pyExport FalkorDB + Qdrant to S3/localboto3, falkordb, qdrant-client
recover_from_qdrant.pyRebuild FalkorDB graph from Qdrant payloadsfalkordb, qdrant-client
restore_from_backup.pyRestore from compressed JSON backupsfalkordb, qdrant-client
health_monitor.pyDetect count drift between databasesrequests, optional webhook client

All scripts connect directly to FalkorDB and Qdrant, bypassing the Flask API. They read the same environment variables as app.py. The backup_automem.py script runs every 6 hours via a GitHub Actions workflow.

Environment Variable Load Order (automem/config.py)

Section titled “Environment Variable Load Order (automem/config.py)”
  1. Load .env from repository root
  2. Load ~/.config/automem/.env (user-specific overrides)
  3. Railway injects secrets as environment variables
CategoryExample VariablesDefined In
Database ConnectionsFALKORDB_HOST, QDRANT_URLautomem/config.py
AuthenticationAUTOMEM_API_TOKEN, ADMIN_API_TOKENautomem/config.py
Consolidation IntervalsCONSOLIDATION_DECAY_INTERVAL_SECONDSautomem/config.py
Enrichment TuningENRICHMENT_MAX_ATTEMPTS, ENRICHMENT_SIMILARITY_THRESHOLDautomem/config.py
Embedding BatchingEMBEDDING_BATCH_SIZE, EMBEDDING_BATCH_TIMEOUT_SECONDSautomem/config.py
Search WeightsSEARCH_WEIGHT_VECTOR, SEARCH_WEIGHT_KEYWORDautomem/config.py
FilePurposeKey Settings
DockerfileContainer image definitionPython 3.11, installs spaCy model
docker-compose.ymlLocal development stackOrchestrates Flask + FalkorDB + Qdrant
railway.jsonRailway deployment configService definitions, port mappings

Docker Compose services:

  • memory-service: Flask API (port 8001)
  • falkordb: Graph database (port 6379)
  • qdrant: Vector database (port 6333)

The test suite covers consolidation, API endpoints, enrichment, embedding providers, and integration flows:

tests/
├── conftest.py
├── test_consolidation_engine.py
├── test_api_endpoints.py
├── test_app.py
├── test_enrichment.py
├── test_embedding_providers.py
├── test_integration.py
├── test_content_size.py
├── test_vector_size_safety.py
├── test_recall_entity_extraction.py
├── support/
├── contracts/
└── benchmarks/

Tests use pytest markers to separate execution tiers: unit (no external services, runs with make test), integration (requires Docker stack, runs with make test-integration), and live (runs against a deployed Railway instance).

Mock objects (tests/test_consolidation_engine.py:12-78):

  • FakeGraph: Implements GraphLike protocol with in-memory state
  • FakeVectorStore: Implements VectorStoreProtocol for deletion tracking
  • FakeResult: Mimics FalkorDB query result structure

Test philosophy: Consolidation tests use deterministic mocks and frozen time to ensure reproducible relevance score calculations.

PatternPurposeExamples
*.pyPython modulesapp.py, consolidation.py
test_*.pyPytest test filestest_consolidation_engine.py
*_automem.pyOperational scriptsbackup_automem.py, restore_from_backup.py
*.mdMarkdown documentationREADME.md, OPTIMIZATIONS.md
*.shShell scriptstest-optimizations.sh
.envEnvironment configuration.env, ~/.config/automem/.env
*.ymlYAML configurationdocker-compose.yml

Gitignored patterns: __pycache__/, venv/, .venv/, backups/, node_modules/, .env

Different deployment scenarios use different subsets of the repository:

  1. Railway (Production): Single container running app.py (which imports the automem/ package), consolidation runs in background thread
  2. Docker Compose (Local): Multi-container with separate database services
  3. Bare Metal (Development): Python process + external databases
  4. MCP Bridge: Optional Node.js service for AI platform integration
  5. Scripts: Standalone utilities connecting directly to databases

The mcp-automem package uses a TypeScript project structure with distinct directories for source code, templates, build artifacts, and configuration files.

graph TB
    ROOT["."]

    ROOT --> SRC["src/"]
    ROOT --> TEMPLATES["templates/"]
    ROOT --> PLUGINS["plugins/"]
    ROOT --> DIST["dist/"]
    ROOT --> GITHUB[".github/"]
    ROOT --> HUSKY[".husky/"]

    SRC --> INDEX["index.ts\n(MCP server entry point)"]
    SRC --> CLIENT["automem-client.ts\n(HTTP client)"]
    SRC --> TYPES["types.ts\n(TypeScript interfaces)"]
    SRC --> CLI["cli/"]

    CLI --> SETUP["setup.ts"]
    CLI --> CLAUDECODE["claude-code.ts"]
    CLI --> QUEUE["queue.ts"]
    CLI --> CONFIG["config.ts"]
    CLI --> CURSOR["cursor.ts"]
    CLI --> CODEX["codex.ts"]
    CLI --> OPENCLAW["openclaw.ts"]

    TEMPLATES --> TCCLI["claude-code/\nhooks, scripts, settings"]
    TEMPLATES --> TCUR["cursor/\n.mdc rule files"]
    TEMPLATES --> TCODEX["codex/\nAGENTS.md"]
    TEMPLATES --> TOCLAW["openclaw/\nSKILL.md"]
    TEMPLATES --> TWARP["warp/\nwarp-rules.md"]

    PLUGINS --> PCCLI[".claude-plugin/\nmarketplace files"]
    PLUGINS --> PSERVER["server/\nserver.json metadata"]

    GITHUB --> WORKFLOWS["workflows/\nCI/CD YAML files"]

    DIST --> DISTJS["*.js\n(Compiled code)"]
    DIST --> DISTDTS["*.d.ts\n(Type definitions)"]
    DIST --> DISTMAP["*.js.map\n(Source maps)"]

The src/index.ts file serves dual purposes based on command-line arguments:

  1. Server Mode (no arguments): Launches an MCP server using StdioServerTransport from @modelcontextprotocol/sdk
  2. CLI Mode (with arguments): Routes commands to appropriate CLI handlers in src/cli/

The src/automem-client.ts file implements the AutoMemClient class, providing a typed HTTP interface to the AutoMem backend service.

MethodHTTP EndpointPurpose
storeMemory()POST /memoryStore new memory with content, tags, metadata
recallMemory()GET /recallHybrid search (vector + keyword + tags)
recallMemoryByTag()GET /memory/by-tagTag-only filtering
associateMemories()POST /associateCreate typed relationships
updateMemory()PATCH /memory/:idUpdate existing memory fields
deleteMemory()DELETE /memory/:idRemove memory and embedding
checkHealth()GET /healthDatabase connectivity status

Configuration resolution (priority order):

  1. Constructor parameters (if provided)
  2. Environment variables (AUTOMEM_ENDPOINT, AUTOMEM_API_KEY)
  3. .env file (loaded via dotenv)

The src/types.ts file defines TypeScript interfaces for all data structures:

  • Configuration types: AutoMemConfig, MCPServerConfig, ClaudeCodeConfig
  • Memory operation arguments: StoreMemoryArgs, RecallMemoryArgs, AssociateMemoriesArgs
  • API response types: RecallResult, MemoryMetadata, RelationshipType
  • CLI types: SetupAnswers, QueueEntry

Each CLI command is implemented as a separate module with a main run*() function. CLI handlers follow a consistent pattern: --dry-run flag for safe preview, --verbose flag for detailed logging, template rendering from templates/ directory, filesystem operations with error handling, and user-friendly output with next-step instructions.

The templates/ directory contains platform-specific integration files that are copied to user systems during installation. These files are included in the published npm package.

templates/
├── claude-code/
│ ├── hooks/ # Shell scripts triggered by Claude events
│ │ ├── PostToolUse.sh # After tool execution
│ │ └── Stop.sh # Session end (queue processor)
│ ├── scripts/ # Support utilities
│ │ ├── session-memory.sh
│ │ ├── memory-filters.json
│ │ └── queue-processor.sh
│ ├── settings.json # Base hook configuration
│ └── profiles/
│ ├── settings.lean.json
│ └── settings.extras.json
├── cursor/
│ ├── automem.mdc # Cursor rule template
│ └── mcp-config.json # MCP server config snippet
├── codex/
│ ├── AGENTS.md # Codex agent rules
│ └── config.toml.snippet # Server config for ~/.codex/
├── openclaw/
│ └── SKILL.md # OpenClaw skill definition
└── warp/
└── warp-rules.md # Warp terminal rules

Template file types:

TypeExtensionPurposeTarget Platform
Hook Scripts.shEvent-driven automationClaude Code
Rule Files.md, .mdcAI assistant instructionsCursor, Codex, Warp, OpenClaw
Config Snippets.json, .tomlMCP server configurationAll platforms
Settings.jsonHook permissions & matchersClaude Code
Filter Configs.jsonMemory significance rulesClaude Code

The dist/ directory is generated by TypeScript compilation (tsc) and contains:

  1. JavaScript files (.js) — Compiled from TypeScript source
  2. Type declarations (.d.ts) — For TypeScript consumers
  3. Source maps (.js.map) — For debugging compiled code

TypeScript configuration (tsconfig.json):

  • outDir: "./dist" — Output location
  • declaration: true — Generate .d.ts files
  • sourceMap: true — Generate source maps
  • module: "ES2020" — ES module format
  • target: "ES2020" — JavaScript target version

Build scripts:

ScriptCommandPurpose
buildtsc && npm run postbuildCompile + make executable
postbuildchmod +x dist/index.jsAdd execute permission
devtsx watch src/index.tsHot-reload development
typechecktsc --noEmitValidate types without building

The plugins/ directory contains a packaged version of the MCP server for Claude Desktop’s plugin system. This directory duplicates content from templates/ to support the .mcpb extension format, which requires a self-contained plugin directory with all resources included.

The repository root contains multiple configuration files:

FilePurpose
package.jsonnpm scripts, dependencies, package metadata
tsconfig.jsonTypeScript compiler options
eslint.config.jsESLint flat config (ESLint 9+)
.commitlintrc.cjsConventional commit message validation
.prettierrcCode formatting rules
release-please-config.jsonAutomated release configuration
.release-please-manifest.jsonCurrent version tracking
env.exampleTemplate for required environment variables

Five files must maintain version consistency (managed by release-please):

  1. package.json"version": "0.12.0"
  2. plugins/mcp-automem/server.json"version": "0.12.0"
  3. manifest.json"version": "0.12.0"
  4. plugin.json"version": "0.12.0"
  5. marketplace.json"version": "0.12.0"

The release-please.yml workflow automatically updates all five files when creating releases based on conventional commits.

WorkflowTriggerPurpose
ci.ymlPR, push to mainLint, build, test, coverage
security.ymlPR, push, scheduleCodeQL, npm audit
release-please.ymlPush to mainVersion bump, changelog, publish
semantic-pr-title.ymlPR open/editValidate PR title format

The .husky/commit-msg hook runs commitlint to validate commit messages against Conventional Commits format before allowing the commit.

Accepted types: fix, feat, chore, docs, refactor, test, ci, build, perf, revert

The files array in package.json specifies what gets included in the npm package:

What’s included: dist/, templates/, plugins/, env.example, README.md, CHANGELOG.md

What’s excluded: Source code (src/), tests, development configs, CI/CD files, .tsbuildinfo

TaskFiles to Modify
Add a new MCP toolsrc/index.ts (schema + handler), src/automem-client.ts (client method), src/types.ts
Add a new CLI commandCreate src/cli/new-command.ts, import and route in src/index.ts
Modify platform integrationEdit templates/<platform>/, update src/cli/<platform>.ts, sync to plugins/
Change build outputModify tsconfig.json, update package.json main and types fields
Add release automationEdit .github/workflows/release-please.yml, update .release-please-manifest.json