Test Query Set - US-015¶

Overview¶

Comprehensive test query set with 50 queries covering all query modes and features of the Graffold.

File Location¶

data/test_queries.yaml

Structure¶

Categories (10 queries each)¶

Multi-Hop Queries - Test variable-length path traversal
Hop depths: 1-3
Example: "What pathways connect BRCA1 to breast cancer?"
Tests: Cypher variable-length paths, relationship chains, intermediate nodes
Global Queries - Test community summary retrieval
Keywords: "overview", "summarize", "landscape"
Example: "Give me an overview of cardiovascular disease biomarkers"
Tests: Community node retrieval, relevance ranking, broad context
Local Queries - Test entity neighborhood search
Focus: Specific entities (TP53, BRCA1, IL-6)
Example: "What is TP53?"
Tests: Hybrid retrieval (vector + fulltext + graph), seed entity expansion
Hybrid Queries - Test combined retrieval strategies
Pattern: Global context + local detail
Example: "What are cardiovascular biomarkers and how does troponin relate to heart failure?"
Tests: Mode mixing, result deduplication, breadth + depth
Disambiguation Queries - Test entity resolution
Aliases: p53 → TP53, tumor necrosis factor → TNF
Example: "What is the role of p53 in cancer?"
Tests: UniProt/MONDO mapping, synonym expansion, canonical ID resolution

Query Fields¶

Each query includes: - query: The natural language question - expected_mode: Expected query mode (local, global, hybrid, naive) - description: What the query tests - hop_depth: (multi-hop only) Path traversal depth - canonical_entity: (disambiguation only) Expected canonical ID - disambiguation_needed: (disambiguation only) Flag for entity resolution

Validation¶

Run validation script:

python scripts/validate_test_queries.py

Expected output:

✅ VALIDATION PASSED - All acceptance criteria met
   - 50 queries total
   - 10 queries per category
   - All required fields present

Usage¶

Load queries in Python:¶

import yaml

with open("data/test_queries.yaml") as f:
    queries = yaml.safe_load(f)

# Access by category
multi_hop = queries["multi_hop_queries"]
global_queries = queries["global_queries"]
local_queries = queries["local_queries"]
hybrid_queries = queries["hybrid_queries"]
disambiguation = queries["disambiguation_queries"]

# Iterate through all queries
for query_obj in multi_hop:
    question = query_obj["query"]
    mode = query_obj["expected_mode"]
    hop_depth = query_obj.get("hop_depth", 1)
    # ... test query

Test with API:¶

# Multi-hop query
curl -X POST "http://localhost:8000/v1/sessions/{session_id}/query" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What pathways connect BRCA1 to breast cancer?",
    "hop_depth": 2
  }'

# Global query
curl -X POST "http://localhost:8000/v1/sessions/{session_id}/query" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Give me an overview of cardiovascular disease biomarkers",
    "mode": "global"
  }'

Acceptance Criteria Status¶

✅ Create 10 multi-hop queries
✅ Create 10 global queries
✅ Create 10 local queries
✅ Create 10 hybrid queries
✅ Create 10 disambiguation queries
✅ Store in data/test_queries.yaml with manual labels

Files Created¶

data/test_queries.yaml - Main test query set (283 lines)
scripts/validate_test_queries.py - Validation script
docs/TEST_QUERY_SET.md - This documentation

Next Steps¶

Use this query set for: 1. Automated testing of query mode classification 2. Performance benchmarking across modes 3. Accuracy evaluation of entity disambiguation 4. Multi-hop traversal validation 5. Community summary retrieval testing