VIII. Conclusion
Knowledge graphs are only as useful as their ontologies allow. When schemas become outdated, valuable information is lost—discarded because it doesn't fit, or force-fitted into inappropriate categories. Traditional approaches force a choice between quality (manual evolution) and scale (automated evolution), but neither alone is satisfactory.
This paper presented "Ontology in the Loop"—a framework for AI-assisted ontology evolution that combines the best of both approaches. The key insight is separating proposal generation (which AI can do at scale) from decision making (which benefits from human judgment).
Summary of Contributions
The framework makes four main contributions:
- Two-Layer LLM Architecture. By separating domain detection from extraction/analysis, we reduce context requirements by ~70%, making the approach practical for large ontologies.
- Domain-Based Schema Slicing. Organizing ontologies into semantic domains enables efficient context management. The LLM sees only relevant schema elements, improving both accuracy and cost efficiency.
- Structured Proposal Workflow. From gap detection through validation to batch application, the workflow ensures proposals are well-formed, non-redundant, and properly reviewed before modifying the production schema.
- Embedding-Based Validation. Vector similarity prevents ontology bloat by catching semantically equivalent proposals before they reach human reviewers.
Design Philosophy
Several principles guided the framework design:
Human authority over schema changes. The framework assists humans but never bypasses them. Every schema modification requires explicit approval. This preserves accountability and catches errors that automated validation might miss.
AI handles volume; humans handle judgment. Processing thousands of documents for potential gaps is tedious; deciding whether a proposed type is semantically appropriate requires understanding. The division of labor plays to each party's strengths.
Precision over recall. False negatives (missed gaps) can be caught in future documents. False positives (bad proposals) waste human time and erode trust. The framework optimizes for high-quality proposals.
Separation of concerns. Production data stays clean; proposals live separately until approved. This prevents pollution and maintains clear audit trails.
Looking Forward
This paper presents a conceptual framework. The natural next step is implementation and experimental validation: measuring actual approval rates, gap detection accuracy, and reviewer throughput with real data.
Beyond validation, several extensions warrant exploration: learning from rejection feedback to improve future proposals, auto-merging similar pending proposals, and confidence-based routing to optimize reviewer allocation.
The broader vision is knowledge graphs that evolve gracefully—adapting to new domains, new concepts, and new relationships without sacrificing the quality control that makes them trustworthy. By keeping humans in the loop while leveraging AI for scale, we can build systems that are both adaptive and reliable.