Knowledge Graph Creation Guide: From Raw Data to Query-Ready Graph¶

Complete workflow for building a production-ready knowledge graph with non-redundant consolidated relationships

Last Updated: 2025-11-21

Table of Contents¶

Overview
Prerequisites
Phase 1: Data Preparation
Phase 2: Foundation - CSV Data Ingestion
Phase 3: Enrichment - PubMed Abstracts
Phase 4: Entity Consolidation
Phase 5: Relationship Consolidation
Phase 6: Vector Embeddings
Phase 7: Validation & Quality
Querying the Graph
Troubleshooting

Overview¶

This guide walks through building a production-quality knowledge graph with: - Non-redundant entities: Multiple mentions consolidated into single canonical nodes - Feature-rich relationships: All associations between node pairs merged with preserved metadata - Biological accuracy: Disease ontologies (MONDO) and protein dictionaries (UniProt) for standardization - Query-ready: Vector embeddings for semantic search

What You'll Build¶

Disease Ontology CSV          Protein Dictionaries CSV
        ↓                              ↓
    [Ingest Foundation Data with IDs & Synonyms]
                    ↓
        [PubMed Abstract Extraction]
                    ↓
        [Entity Consolidation Pipeline]
         - UniProt ID matching
         - Synonym/gene symbol matching  
         - Fuzzy name matching
                    ↓
        [Relationship Consolidation]
         - Merge duplicate relationships
         - Preserve all features/metadata
         - Track provenance
                    ↓
        [Generate Vector Embeddings]
                    ↓
        [Query-Ready Knowledge Graph]

Prerequisites¶

Software Requirements¶

# Python 3.10+
python --version

# Install dependencies
pip install uv
uv sync

Database Setup¶

Graffold supports both Neo4j and Memgraph. Choose one:

Neo4j (Recommended for production)

# Install Neo4j Desktop or Docker
docker run -d \
  --name neo4j \
  -p 7474:7474 -p 7687:7687 \
  -e NEO4J_AUTH=neo4j/your_password \
  neo4j:5.13

Memgraph (Lightweight alternative)

docker compose -f docker-compose.yml -f docker-compose.memgraph.yml --profile memgraph up -d

Environment Configuration¶

Create .env file:

# Database (works with Neo4j, Memgraph, and FalkorDB)
DATABASE_TYPE=neo4j  # or "memgraph" or "falkordb"
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_password

# LLM Service
OPENAI_API_KEY=sk-...  # If using OpenAI
# Or configure AWS SageMaker endpoint for local models

Phase 1: Data Preparation¶

Organize Your Source Data¶

data/
├── ontologies/
│   ├── mondo_disease_ontology.csv          # MONDO disease IDs, names, hierarchies
│   └── disease_synonyms.csv                # Alternative disease names
├── proteins/
│   ├── uniprot_protein_dictionary.csv      # UniProt IDs, names, gene symbols
│   ├── protein_synonyms.csv                # Alternative protein names
│   └── gene_symbols.csv                    # Gene to protein mappings
└── pubmed/
    └── queries.txt                          # Search terms for abstract retrieval

Expected CSV Formats¶

Disease Ontology CSV (mondo_disease_ontology.csv):

mondo_id,disease_name,synonyms,parent_id,description
MONDO:0005148,Type 2 Diabetes,"T2D|Diabetes Mellitus Type 2|NIDDM",MONDO:0005015,Non-insulin dependent diabetes
MONDO:0005267,Heart Disease,"Cardiac Disease|Cardiovascular Disease",MONDO:0005385,Diseases of the heart

Protein Dictionary CSV (uniprot_protein_dictionary.csv):

uniprot_id,protein_name,gene_symbol,synonyms,function
P04637,TP53,TP53,"Tumor Protein P53|p53|Antigen NY-CO-13",Tumor suppressor
P38398,BRCA1,BRCA1,"Breast Cancer Type 1|RING Finger Protein 53",DNA repair

Protein Synonyms CSV (protein_synonyms.csv):

uniprot_id,synonym,synonym_type
P04637,p53,common_name
P04637,Tumor Protein P53,full_name
P04637,Antigen NY-CO-13,alias

Phase 2: Foundation - CSV Data Ingestion¶

Goal: Create canonical entities with proper IDs and synonyms before adding literature data.

Step 2.1: Ingest Disease Ontology¶

export DATABASE="olink1"

# Ingest disease ontology with MONDO IDs
uv run python enrichment_main.py \
  --file data/ontologies/mondo_disease_ontology.csv \
  --database $DATABASE \
  --column-handlers "0:mondo-id,1:disease-name,2:synonyms,3:parent-id,4:prop"

What this creates: - Disease nodes with mondo_id as primary identifier - name property from disease_name column - synonyms property (array) for alternative names - PARENT_OF relationships for disease hierarchy - description as additional property

Verify:

// Check disease nodes
MATCH (d:Disease)
RETURN d.mondo_id, d.name, d.synonyms
LIMIT 10;

// Check hierarchy
MATCH (parent:Disease)-[:PARENT_OF]->(child:Disease)
RETURN parent.name, child.name
LIMIT 10;

Step 2.2: Ingest Protein Dictionaries¶

# Ingest primary protein dictionary
uv run python enrichment_main.py \
  --file data/proteins/uniprot_protein_dictionary.csv \

  --database $DATABASE \
  --column-handlers "0:uniprot-id,1:protein-name,2:gene-symbol,3:synonyms,4:prop"

# Ingest additional synonym mappings
uv run python enrichment_main.py \
  --file data/proteins/protein_synonyms.csv \

  --database $DATABASE \
  --column-handlers "0:uniprot-id,1:synonym,2:synonym-type"

What this creates: - Protein nodes with uniprot_id as primary identifier - name property from protein_name - gene_symbol property for gene mapping - synonyms property (array) for alternative names - function property for protein description

Verify:

// Check protein nodes
MATCH (p:Protein)
RETURN p.uniprot_id, p.name, p.gene_symbol, p.synonyms
LIMIT 10;

// Count unique proteins
MATCH (p:Protein)
WHERE p.uniprot_id IS NOT NULL
RETURN count(DISTINCT p.uniprot_id) as unique_proteins;

Step 2.3: Create Indexes for Performance¶

# Create indexes on identifier properties
uv run python -m src.ingest_pipeline.embedding_pipeline \
  --database $DATABASE \

  --operation create-indexes

What this creates: - Vector index on embedding property (for semantic search) - Fulltext index on name, synonyms, text properties (for keyword search) - Range indexes on mondo_id, uniprot_id (for fast lookups)

Phase 3: Enrichment - PubMed Abstracts¶

Goal: Extract protein-disease associations from scientific literature.

Step 3.1: Retrieve PubMed Abstracts¶

# Ingest abstracts for your domain
uv run python ingest_main.py \
  --search-term "cardiovascular disease protein biomarker" \
  --max-results 1000 \
  --database $DATABASE \

  --service local \

What this extracts: - Protein entities mentioned in abstracts - Disease entities mentioned in abstracts - ASSOCIATES_WITH relationships between them - Provenance: PMID, abstract text, extraction confidence

Key Parameters: - --max-results: Number of PubMed abstracts to retrieve - --service: LLM service (local, openai, sagemaker-llama3)

Step 3.2: Multiple Search Terms¶

# Create query file
cat > data/pubmed/queries.txt << EOF
cardiovascular disease protein biomarker
heart failure inflammatory markers
diabetes mellitus protein expression
cancer tumor suppressor proteins
EOF

# Batch ingest
while IFS= read -r query; do
  echo "Processing: $query"
  uv run python ingest_main.py \
    --search-term "$query" \
    --max-results 500 \
    --database $DATABASE \

    --service local \

done < data/pubmed/queries.txt

After ingestion, you'll have: - Raw entities: Many duplicate protein/disease mentions with slight name variations - Unlinked data: Entities from PubMed not yet matched to canonical CSV entities - Unmerged relationships: Multiple relationships between same protein-disease pairs

Example of the problem:

// Multiple nodes for same protein
MATCH (p:Protein) WHERE p.name CONTAINS "TP53"
RETURN p.name;
// Results: "TP53", "p53", "Tumor Protein P53", "tumor protein p53"

// Multiple relationships between same pair
MATCH (p:Protein {name:"TP53"})-[r:ASSOCIATES_WITH]->(d:Disease {name:"Cancer"})
RETURN count(r);
// Results: 47 relationships (should be 1 consolidated relationship)

Phase 4: Entity Consolidation¶

Goal: Merge duplicate entities into single canonical nodes with all properties preserved.

Overview: Multi-Strategy Consolidation¶

We use a cascading consolidation pipeline with multiple strategies:

UniProt ID Matching (highest confidence)
Synonym/Gene Symbol Matching (high confidence)
Fuzzy Name Matching (medium confidence)
LLM-based Matching (for ambiguous cases)

Step 4.1: UniProt ID-Based Consolidation¶

Most accurate: Merges proteins with same UniProt ID.

uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation consolidate-uniprot

What this does:

// Before
(:Protein {name:"TP53", uniprot_id:"P04637"})
(:Protein {name:"p53", uniprot_id:"P04637"})
(:Protein {name:"Tumor Protein P53", uniprot_id:"P04637"})

// After
(:Protein {
  name:"TP53",  // Primary name
  uniprot_id:"P04637",
  synonyms:["p53", "Tumor Protein P53"],  // Merged
  gene_symbol:"TP53",
  consolidated_count:3  // Track merge source
})

Log output:

✓ Consolidated 1,247 proteins by UniProt ID
✓ Merged 3,891 relationships
✓ Preserved all properties and provenance

Step 4.2: Synonym & Gene Symbol Consolidation¶

High accuracy: Merges entities sharing gene symbols or synonyms from CSV dictionaries.

uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation consolidate-name-features

What this does:

// Before
(:Protein {name:"BRCA1", gene_symbol:"BRCA1"})
(:Protein {name:"Breast Cancer Type 1", gene_symbol:"BRCA1"})
(:Protein {name:"RING Finger Protein 53"})  // synonym of BRCA1

// After  
(:Protein {
  name:"BRCA1",
  gene_symbol:"BRCA1",
  synonyms:["Breast Cancer Type 1", "RING Finger Protein 53"],
  consolidated_count:3
})

Step 4.3: Fuzzy Name-Based Consolidation¶

Medium accuracy: Merges similar names using Levenshtein distance and semantic similarity.

uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation consolidate \
  --similarity-threshold 0.85

What this handles: - Typos: "Interleukin 6" ≈ "Interleukin-6" ≈ "interleukin 6" - Abbreviations: "IL-6" ≈ "IL6" - Minor variations: "Type 2 Diabetes" ≈ "Diabetes Type 2"

Parameters: - --similarity-threshold: 0.0 (no match) to 1.0 (exact match). Default: 0.85 - Lower threshold = more aggressive merging (risk of false positives) - Higher threshold = conservative merging (may miss true duplicates)

Step 4.4: Disease MONDO ID Labeling¶

Adds canonical disease identifiers:

uv run python -m src.processors.ontology_filter \
  --database $DATABASE \

  --operation label_diseases

What this does: - Matches disease names to MONDO ontology - Adds mondo_id property to disease nodes - Creates MONDO_CLASSIFICATION relationships - Enables disease hierarchy queries

Step 4.5: Protein UniProt ID Labeling¶

Adds canonical protein identifiers:

uv run python -m src.processors.ontology_filter \
  --database $DATABASE \

  --operation label_proteins

What this does: - Matches protein names/gene symbols to UniProt - Adds uniprot_id property to protein nodes - Links to gene ontology terms - Enables protein family queries

Step 4.6: Comprehensive Pipeline (Recommended)¶

Run all consolidation steps in one command:

uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation full

This executes: 1. UniProt ID consolidation 2. Name features consolidation 3. Traditional fuzzy matching 4. MONDO ID labeling 5. UniProt ID labeling 6. Quality validation checks

Expected results:

=== Entity Consolidation Summary ===
Starting entities: 12,453 proteins, 3,891 diseases
After consolidation: 8,247 proteins (-34%), 2,156 diseases (-45%)
Relationships: 47,239 raw → 19,834 consolidated (-58%)
Quality metrics:
  - Nodes with IDs: 98.3% proteins, 94.7% diseases
  - Synonym coverage: 87.2%
  - Provenance tracking: 100%

Phase 5: Relationship Consolidation¶

Goal: Merge duplicate relationships between the same node pairs while preserving all features.

The Problem¶

After entity consolidation, you'll have multiple relationships between same entities:

// Before relationship consolidation
MATCH (p:Protein {name:"TP53"})-[r:ASSOCIATES_WITH]->(d:Disease {name:"Cancer"})
RETURN 
  r.pmid,
  r.confidence,
  r.abstract_text,
  r.extraction_method
// Returns 47 relationships with different PMIDs, confidence scores, abstracts

Issues: - Graph clutter: 47 edges instead of 1 - Inefficient queries: Must scan all duplicate relationships - Lost context: Hard to see aggregate evidence - Visualization problems: Overwhelmed by duplicate edges

Step 5.1: Relationship Consolidation with Feature Preservation¶

uv run python ingest_main.py \
  --consolidate-relationships \
  --database $DATABASE \

  --relationship-types "ASSOCIATES_WITH,INTERACTS_WITH,REGULATES"

What this does:

// After relationship consolidation
MATCH (p:Protein {name:"TP53"})-[r:ASSOCIATES_WITH]->(d:Disease {name:"Cancer"})
RETURN r
// Returns 1 relationship with:
{
  relationship_count: 47,  // Number of source relationships
  pmids: ["12345678", "23456789", ...],  // All PMIDs (array)
  confidence_scores: [0.95, 0.87, 0.91, ...],  // All scores
  avg_confidence: 0.89,  // Computed average
  max_confidence: 0.95,  // Highest confidence
  min_confidence: 0.78,  // Lowest confidence
  extraction_methods: ["llm", "rule-based", ...],  // All methods
  abstract_excerpts: ["TP53 mutations found in...", ...],  // Key excerpts
  first_seen: "2023-01-15",  // Earliest evidence
  last_seen: "2024-11-20",  // Latest evidence
  evidence_sources: 47  // Total evidence count
}

Key features preserved: - ✅ All PMIDs (provenance) - ✅ All confidence scores (with statistics) - ✅ All extraction methods - ✅ Representative text excerpts - ✅ Temporal information (first/last seen) - ✅ Evidence count

Step 5.2: Verify Consolidation Quality¶

# Check consolidation statistics
uv run python -m src.processors.relationship_counter \
  --database $DATABASE \

  --operation stats

Expected output:

=== Relationship Consolidation Report ===

ASSOCIATES_WITH:
  Before: 47,239 relationships
  After: 12,834 relationships (-73%)
  Avg features per relationship: 3.7
  Max evidence sources: 23

INTERACTS_WITH:
  Before: 8,923 relationships
  After: 3,456 relationships (-61%)
  Avg features per relationship: 2.6
  Max evidence sources: 15

Quality Metrics:
  ✓ All relationships have evidence_sources property
  ✓ 98.7% have pmids array
  ✓ 94.3% have confidence scores
  ✓ No orphaned relationships

Step 5.3: Query Consolidated Relationships¶

// Find strongest protein-disease associations
MATCH (p:Protein)-[r:ASSOCIATES_WITH]->(d:Disease)
WHERE r.evidence_sources >= 5  // At least 5 independent sources
  AND r.avg_confidence >= 0.85  // High average confidence
RETURN 
  p.name as protein,
  d.name as disease,
  r.evidence_sources as sources,
  r.avg_confidence as confidence,
  r.pmids[0..3] as sample_pmids  // First 3 PMIDs
ORDER BY r.evidence_sources DESC
LIMIT 20;

// Find most studied proteins
MATCH (p:Protein)-[r:ASSOCIATES_WITH]->()
RETURN 
  p.name,
  count(r) as disease_associations,
  sum(r.evidence_sources) as total_evidence
ORDER BY total_evidence DESC
LIMIT 10;

Step 5.4: Advanced Consolidation Options¶

Confidence weighting:

uv run python ingest_main.py \
  --consolidate-relationships \
  --database $DATABASE \

  --weight-by-confidence \
  --min-confidence 0.7

Time-aware consolidation:

uv run python ingest_main.py \
  --consolidate-relationships \
  --database $DATABASE \

  --time-window "2020-01-01:2024-12-31" \
  --temporal-decay

Phase 6: Vector Embeddings¶

Goal: Generate semantic embeddings for hybrid search (vector + keyword + graph).

Step 6.1: Generate Embeddings¶

CRITICAL for search functionality:

uv run python ingest_main.py \
  --add-graph-embeddings \
  --database $DATABASE \

  --service local

What this creates: - embedding property on all nodes (768-dimensional vector) - Vector index for similarity search - Enables semantic queries: "Find proteins similar to TP53"

Embedding model (default: all-mpnet-base-v2): - Dimensions: 768 - Context: General biomedical text - Performance: Good balance of speed and accuracy

Step 6.2: Verify Embeddings¶

// Check embedding coverage
MATCH (n)
WHERE n.embedding IS NOT NULL
RETURN labels(n)[0] as node_type, count(n) as with_embeddings;

// Test similarity search
CALL db.index.vector.queryNodes(
  'node_embeddings',
  5,
  [/* TP53 embedding vector */]
) YIELD node, score
RETURN node.name, score;

Step 6.3: Alternative Embedding Models¶

For specific domains:

# BioBERT (optimized for biomedical text)
uv run python ingest_main.py \
  --add-graph-embeddings \
  --database $DATABASE \

  --embedding-model "dmis-lab/biobert-base-cased-v1.2"

# PubMedBERT (trained on PubMed abstracts)
uv run python ingest_main.py \
  --add-graph-embeddings \
  --database $DATABASE \

  --embedding-model "microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract"

Phase 7: Validation & Quality¶

Step 7.1: Quality Checks¶

# Run comprehensive quality checks
uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation validate

Checks performed: - ✓ Node ID coverage (% with uniprot_id/mondo_id) - ✓ Synonym completeness - ✓ Relationship consolidation - ✓ Embedding coverage - ✓ Orphaned nodes - ✓ Data consistency

Expected report:

=== Knowledge Graph Quality Report ===

Entity Quality:
  Proteins: 8,247 total
    ✓ 98.3% have uniprot_id
    ✓ 87.2% have synonyms
    ✓ 100% have embeddings
    ⚠ 1.7% missing uniprot_id (142 nodes)

  Diseases: 2,156 total
    ✓ 94.7% have mondo_id
    ✓ 82.1% have synonyms
    ✓ 100% have embeddings
    ⚠ 5.3% missing mondo_id (114 nodes)

Relationship Quality:
  ✓ 12,834 ASSOCIATES_WITH (consolidated)
  ✓ 3,456 INTERACTS_WITH (consolidated)
  ✓ Avg 3.7 evidence sources per relationship
  ✓ No orphaned relationships
  ✓ All relationships have provenance

Graph Statistics:
  Total nodes: 10,403
  Total relationships: 16,290
  Avg degree: 3.1
  Density: 0.00031
  Connected components: 1 (fully connected)

Step 7.2: Fix Common Issues¶

Missing UniProt IDs:

# Attempt to map remaining proteins
uv run python -m src.processors.ontology_filter \
  --database $DATABASE \

  --operation label_proteins \
  --fuzzy-match \
  --confidence-threshold 0.8

Missing MONDO IDs:

# Attempt to map remaining diseases
uv run python -m src.processors.ontology_filter \
  --database $DATABASE \

  --operation label_diseases \
  --fuzzy-match \
  --confidence-threshold 0.8

Orphaned nodes:

// Find nodes with no relationships
MATCH (n)
WHERE NOT (n)--()
RETURN labels(n)[0] as type, count(n) as count;

// Delete if appropriate
MATCH (n)
WHERE NOT (n)--()
DELETE n;

Step 7.3: Performance Optimization¶

# Create additional indexes for common queries (Cypher works on both Neo4j and Memgraph)
uv run python << EOF
from neo4j import GraphDatabase

driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))

with driver.session() as session:
    # Index on relationship properties
    session.run("CREATE INDEX rel_evidence IF NOT EXISTS FOR ()-[r:ASSOCIATES_WITH]-() ON (r.evidence_sources)")
    session.run("CREATE INDEX rel_confidence IF NOT EXISTS FOR ()-[r:ASSOCIATES_WITH]-() ON (r.avg_confidence)")

    # Composite indexes for complex queries
    session.run("CREATE INDEX protein_name_uniprot IF NOT EXISTS FOR (p:Protein) ON (p.name, p.uniprot_id)")
    session.run("CREATE INDEX disease_name_mondo IF NOT EXISTS FOR (d:Disease) ON (d.name, d.mondo_id)")

driver.close()
print("✓ Performance indexes created")
EOF

Querying the Graph¶

Web Interface¶

# Start Streamlit app
uv run streamlit run streamlit_app.py

Navigate to http://localhost:8501 and try: - "What proteins are associated with heart disease?" - "Show me proteins that interact with TP53" - "Find diseases related to inflammatory responses"

REST API¶

# Start API server
uv run granian --interface asgi src.api.app:app --host 127.0.0.1 --port 8000 --reload

# Create session
curl -X POST "http://localhost:8000/v1/sessions" \
  -H "Content-Type: application/json" \
  -d '{
    "llm_service": "local",
    "database_name": "olink1",
    "database_type": "neo4j",
    "agent_type": "graph_rag_agent",
    "use_refrag": true,
    "connection_details": {
      "uri": "bolt://localhost:7687",
      "user": "neo4j",
      "password": "your_password"
    }
  }'

# Query (replace {session_id} with response from above)
curl -X POST "http://localhost:8000/v1/sessions/{session_id}/query" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What proteins are associated with cardiovascular disease?",
    "use_refrag": true
  }'

Cypher Queries¶

Consolidated relationship with all evidence:

MATCH (p:Protein {name:"TP53"})-[r:ASSOCIATES_WITH]->(d:Disease)
RETURN 
  d.name as disease,
  r.evidence_sources as evidence_count,
  r.avg_confidence as confidence,
  r.pmids as supporting_papers,
  size(r.pmids) as paper_count
ORDER BY r.evidence_sources DESC;

Proteins by evidence strength:

MATCH (p:Protein)-[r:ASSOCIATES_WITH]->(d:Disease)
WHERE r.evidence_sources >= 10
WITH p, 
  count(d) as disease_count,
  sum(r.evidence_sources) as total_evidence,
  avg(r.avg_confidence) as avg_confidence
RETURN 
  p.name,
  p.uniprot_id,
  disease_count,
  total_evidence,
  round(avg_confidence, 2) as avg_confidence
ORDER BY total_evidence DESC
LIMIT 20;

Network analysis:

// Find protein-disease-protein triangles (shared disease associations)
MATCH (p1:Protein)-[r1:ASSOCIATES_WITH]->(d:Disease)<-[r2:ASSOCIATES_WITH]-(p2:Protein)
WHERE id(p1) < id(p2)  // Avoid duplicates
  AND r1.evidence_sources >= 5
  AND r2.evidence_sources >= 5
RETURN 
  p1.name as protein1,
  p2.name as protein2,
  d.name as shared_disease,
  r1.evidence_sources + r2.evidence_sources as total_evidence
ORDER BY total_evidence DESC
LIMIT 50;

Troubleshooting¶

Issue: Low Consolidation Rate¶

Symptom: Still seeing many duplicate entities after consolidation.

Solutions:

Check ID coverage:

MATCH (p:Protein)
RETURN 
  count(p) as total,
  count(p.uniprot_id) as with_uniprot,
  100.0 * count(p.uniprot_id) / count(p) as coverage_percent;

Run ID labeling first:

# Must be done BEFORE consolidation
uv run python -m src.processors.ontology_filter \
  --database $DATABASE \

  --operation label_proteins

Lower similarity threshold:

uv run python -m src.processors.entity_resolver \
  --database $DATABASE \

  --operation consolidate \
  --similarity-threshold 0.75  # More aggressive

Issue: Missing Relationships After Consolidation¶

Symptom: Relationship count unexpectedly low.

Check:

// Find nodes with no relationships
MATCH (n)
WHERE NOT (n)--()
RETURN labels(n)[0], count(n);

// Verify relationship properties
MATCH ()-[r:ASSOCIATES_WITH]->()
RETURN 
  count(r) as total,
  count(r.evidence_sources) as with_evidence,
  avg(r.evidence_sources) as avg_sources;

Solution: Rerun relationship consolidation:

uv run python ingest_main.py \
  --consolidate-relationships \
  --database $DATABASE \

  --preserve-all-properties

Issue: Slow Query Performance¶

Solutions:

Check indexes:
```
SHOW INDEXES;
```

Add missing indexes:

CREATE INDEX protein_uniprot IF NOT EXISTS FOR (p:Protein) ON (p.uniprot_id);
CREATE INDEX disease_mondo IF NOT EXISTS FOR (d:Disease) ON (d.mondo_id);

Optimize embeddings:

# Rebuild vector index with better parameters
uv run python -m src.ingest_pipeline.embedding_pipeline \
  --database $DATABASE \

  --operation rebuild-vector-index \
  --dimensions 768 \
  --similarity-function cosine

Issue: Embeddings Not Working¶

Symptom: Semantic search returns no results.

Check:

MATCH (n)
WHERE n.embedding IS NULL
RETURN labels(n)[0], count(n);

Solution:

# Regenerate embeddings
uv run python ingest_main.py \
  --add-graph-embeddings \
  --database $DATABASE \

  --service local \
  --force-regenerate

Summary: Complete Workflow Script¶

#!/bin/bash
# Complete KG creation workflow

export DATABASE="olink1"

echo "=== Phase 1: CSV Foundation ==="
# Diseases
uv run python enrichment_main.py \
  --file data/ontologies/mondo_disease_ontology.csv \
  --database $DATABASE \
  --column-handlers "0:mondo-id,1:disease-name,2:synonyms"

# Proteins
uv run python enrichment_main.py \
  --file data/proteins/uniprot_protein_dictionary.csv \
  --database $DATABASE \
  --column-handlers "0:uniprot-id,1:protein-name,2:gene-symbol,3:synonyms"

echo "=== Phase 2: PubMed Abstracts ==="
uv run python ingest_main.py \
  --search-term "cardiovascular disease protein biomarker" \
  --max-results 1000 \
  --database $DATABASE \
  --service local

echo "=== Phase 3: Entity Consolidation ==="
uv run python -m src.processors.entity_resolver \
  --database $DATABASE \
  --operation full

echo "=== Phase 4: Relationship Consolidation ==="
uv run python ingest_main.py \
  --consolidate-relationships \
  --database $DATABASE

echo "=== Phase 5: Vector Embeddings ==="
uv run python ingest_main.py \
  --add-graph-embeddings \
  --database $DATABASE \
  --service local

echo "=== Phase 6: Quality Validation ==="
uv run python -m src.processors.entity_resolver \
  --database $DATABASE \
  --operation validate

echo "✓ Knowledge graph creation complete!"
echo "  Launch queries: uv run streamlit run streamlit_app.py"

Additional Resources¶

ENTITY_CONSOLIDATION.md - Deep dive into consolidation algorithms
RELATIONSHIP_COUNTER.md - Relationship consolidation details
NEO4J_GRAPHRAG_BEST_PRACTICES.md - Integration with neo4j_graphrag library
PPI_FRAMEWORK.md - Protein-protein interaction network analysis

Last updated: 2025-11-21
Questions? Open an issue: https://github.com/graffold/graffold-api/issues