Database Architecture

Graffold uses a split database architecture that separates graph storage from vector search. This decouples the concerns of relationship traversal and semantic similarity, allowing each layer to scale independently.

---

Architecture Overview

flowchart LR
    subgraph APP["Graffold API"]
        QA[Query Agents]
        IS[Ingestion Service]
        SI[SearchIndex Interface]
    end

    subgraph GRAPH["Graph Layer — Memgraph"]
        MG[(Memgraph)]
        MAGE[MAGE Algorithms]
    end

    subgraph VECTOR["Vector Layer — Cloudflare Vectorize"]
        VZ[(Vectorize Index)]
        CF_API[REST API]
    end

    QA -->|Cypher queries| MG
    QA -->|similarity search| SI
    IS -->|nodes & relationships| MG
    IS -->|embeddings| SI
    SI --> CF_API --> VZ
    MG --> MAGE

---

Graph Layer: Memgraph

Role: Stores all nodes, relationships, and properties. Handles Cypher queries, graph traversals, community detection, and path algorithms.

Aspect	Detail
Engine	Memgraph (in-memory graph database)
Image	memgraph/memgraph-mage:latest (includes MAGE algorithm library)
Protocol	Bolt (compatible with Neo4j drivers)
Query language	openCypher
Algorithms	MAGE — PageRank, community detection, shortest path, betweenness centrality
Isolation	Dedicated container per knowledge graph (max/enterprise tiers)
Memory	256 MB–1 GB per instance depending on tier

Why Memgraph over Neo4j?

• In-memory performance — sub-millisecond traversals for interactive queries
• Lower resource footprint — runs comfortably in 256 MB containers
• MAGE algorithms — native graph algorithms without APOC dependency
• Bolt-compatible — same driver code works with Neo4j and Memgraph via DatabaseInterface abstraction
• Container isolation — trivial to spin up per-tenant instances via Docker

Configuration

# Environment variables
DATABASE_TYPE=memgraph
NEO4J_URI=bolt://memgraph:7687   # Uses NEO4J_* vars for Bolt compatibility
NEO4J_USER=""                     # Memgraph default: no auth
NEO4J_PASSWORD=""

Multi-Instance Deployment (Hetzner Dev)

The dev environment runs 5 Memgraph instances for multi-tenant demos:

Instance	Port	Domain
demo0	7687	Default / sandbox
demo1	7688	Demo tenant 1
demo2	7689	Demo tenant 2
demo3	7690	Demo tenant 3
demo4	7691	Biology (primary KG)

---

Vector Layer: Cloudflare Vectorize

Role: Stores and queries vector embeddings for semantic similarity search. Offloads all vector operations from the graph database.

Aspect	Detail
Service	Cloudflare Vectorize
Index name	graffold-embeddings
Dimensions	768 (all-distilroberta-v1 embeddings)
Similarity	Cosine
API	REST (Vectorize v2)
Insert format	NDJSON
Management	Index created via wrangler CLI, not application code

Why Cloudflare Vectorize?

• Memgraph has no native vector indexes — unlike Neo4j's HNSW indexes, Memgraph doesn't support vector search natively
• Edge-deployed — low-latency queries from Cloudflare's global network
• Managed scaling — no infrastructure to maintain for vector search
• Cost-effective — pay-per-query pricing vs. provisioning dedicated vector DB instances
• Decoupled scaling — vector index size grows independently of graph memory

Configuration

# Environment variables
CF_ACCOUNT_ID=<cloudflare-account-id>
CF_API_TOKEN=<cloudflare-api-token>
VECTOR_BACKEND=vectorize          # Tells agents to use Vectorize for vector search

How It Works

1. Ingestion — After entity/relationship extraction, the embedding pipeline generates 768-dimensional vectors for nodes and text chunks, then upserts them to Vectorize via NDJSON POST.
2. Query — When a query agent needs semantic similarity (e.g., "find proteins similar to BRCA1"), it calls CloudflareVectorizeSearchIndex.vector_search() which hits the Vectorize REST API.
3. Hybrid retrieval — Vector results from Vectorize are combined with graph traversal results from Memgraph to produce ranked, contextual answers.

Vector Metadata Schema

Each vector stored in Vectorize carries metadata for result enrichment:

{
  "id": "node-uuid-or-chunk-id",
  "values": [0.012, -0.034, ...],
  "metadata": {
    "node_id": "uuid",
    "name": "BRCA1",
    "label": "Protein",
    "text": "BRCA1 is a tumor suppressor...",
    "source_pmid": "12345678"
  }
}

---

SearchIndex Abstraction

The SearchIndex interface (src/core/search.py) abstracts over all vector/fulltext backends:

# Backend selection at runtime
def create_search_index(driver, backend, database) -> SearchIndex:
    if backend == "vectorize":
        return CloudflareVectorizeSearchIndex()
    if backend == "memgraph":
        return MemgraphSearchIndex(driver)  # fulltext only
    if backend == "falkordb":
        return FalkorDBSearchIndex(driver)
    return Neo4jSearchIndex(driver, database)

When DATABASE_TYPE=memgraph, agents automatically use Cloudflare Vectorize for vector search and Memgraph for fulltext/graph queries. This is configured via the VECTOR_BACKEND=vectorize environment variable or detected automatically when the graph backend is Memgraph.

---

Data Flow: Split Query Path

sequenceDiagram
    participant U as User
    participant A as Query Agent
    participant V as Cloudflare Vectorize
    participant M as Memgraph

    U->>A: "What proteins interact with BRCA1?"
    A->>V: vector_search(embedding("BRCA1"), topK=10)
    V-->>A: [{node_id: "abc", score: 0.95, name: "BRCA1"}, ...]
    A->>M: MATCH (p:Protein {node_id: "abc"})-[r:INTERACTS_WITH]->(t) RETURN t
    M-->>A: [TP53, RAD51, PALB2, ...]
    A->>A: Merge vector + graph results, re-rank
    A-->>U: "BRCA1 interacts with TP53, RAD51, PALB2..."

---

Data Flow: Ingestion

sequenceDiagram
    participant I as Ingestion Pipeline
    participant M as Memgraph
    participant E as Embedding Model
    participant V as Cloudflare Vectorize

    I->>M: MERGE (p:Protein {name: "BRCA1", uniprot_id: "P38398"})
    I->>M: MERGE (d:Disease {name: "Breast Cancer"})
    I->>M: MERGE (p)-[:ASSOCIATED_WITH {confidence: 0.92}]->(d)
    I->>E: encode("BRCA1 tumor suppressor protein")
    E-->>I: [0.012, -0.034, ...]  (768d vector)
    I->>V: POST /insert {id: "node-uuid", values: [...], metadata: {...}}
    V-->>I: {success: true}

---

Backend Compatibility Matrix

Feature	Neo4j	Memgraph	FalkorDB
Graph storage	✅	✅	✅
Cypher queries	✅	✅ (openCypher)	✅
Native vector indexes	✅ (HNSW)	❌	❌
Vector search	Native	CF Vectorize	CF Vectorize
Fulltext indexes	✅	✅ (via MAGE)	✅
Graph algorithms	APOC/GDS	MAGE (built-in)	Limited
Multi-tenant isolation	Label-prefix	Dedicated container	Named graphs
Memory model	Disk + cache	In-memory	In-memory (Redis)
Tier availability	Enterprise	Max / Enterprise	Pro

---

Deployment Topology

Development (Local)

┌─────────────────────────────────────────┐
│  Docker Compose (memgraph profile)      │
│                                         │
│  ┌──────────┐  ┌───────┐  ┌─────────┐  │
│  │ Memgraph │  │ Redis │  │ FastAPI │  │
│  │ :7687    │  │ :6379 │  │ :8000   │  │
│  └──────────┘  └───────┘  └─────────┘  │
└─────────────────────────────────────────┘
         │                        │
         │              ┌─────────┴──────────┐
         │              │ Cloudflare Vectorize│
         │              │ (graffold-embeddings)│
         │              └────────────────────┘
         │
    ┌────┴─────┐
    │ Memgraph │
    │   Lab    │
    │  :3000   │
    └──────────┘

Production (Hetzner CX33)

┌──────────────────────────────────────────────────────┐
│  Hetzner CX33 (4 vCPU, 8 GB RAM)                    │
│                                                      │
│  ┌────────────────────────────────────────────────┐  │
│  │ Caddy (reverse proxy + TLS)                    │  │
│  │   dev.graffold.com → FastAPI                   │  │
│  │   wiki.graffold.com → /opt/graffold/wiki/      │  │
│  └────────────────────────────────────────────────┘  │
│                                                      │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐            │
│  │ demo0    │ │ demo1    │ │ demo2    │ ...×5      │
│  │ :7687    │ │ :7688    │ │ :7689    │            │
│  └──────────┘ └──────────┘ └──────────┘            │
│                                                      │
│  ┌──────────┐  ┌───────┐  ┌──────────┐             │
│  │ FastAPI  │  │ Redis │  │ MG Lab   │             │
│  │ :8000    │  │ :6379 │  │ :3000    │             │
│  └──────────┘  └───────┘  └──────────┘             │
└──────────────────────────────────────────────────────┘
                      │
            ┌─────────┴──────────┐
            │ Cloudflare Vectorize│
            │ (edge-deployed)     │
            └────────────────────┘

---

Fallback Behavior

Scenario	Behavior
Vectorize credentials missing	Vector search returns empty results; fulltext + graph traversal still work
Memgraph container down	Provisioner retries; queries fail with connection error
Neo4j backend selected	Uses native HNSW vector indexes instead of Vectorize
VECTOR_BACKEND not set + Memgraph	Auto-detects and uses Vectorize if CF_ACCOUNT_ID is set