Between releases 2.5.12 and 2.5.14, we permanently removed the Knowledge Graph from Wikantik. The deletion was sweeping: 699 files modified, 15 PostgreSQL tables dropped in Flyway migration V070, 21 ontology relationship predicates retired, and a net reduction of 70,516 lines of Java and TypeScript code. Every accompanying background service—the automated LLM entity extractor, the multi-agent proposal judge, the inclusion policy engine, hub discovery, SHACL validation pipelines, and 15 Model Context Protocol (MCP) tools—has been excised from the repository.
In modern software engineering, deleting a major subsystem that required months of development is rarely done lightly. The rationale for this architectural decision was straightforward and empirical: the Knowledge Graph was not improving search ranking or retrieval quality. Despite its conceptual elegance on paper, the extracted entity graph generated substantial operational latency, compounding synchronization complexity, and noisy traversal paths that consistently underperformed simpler, deterministic retrieval primitives. When an architectural component fails to deliver measurable retrieval gains while imposing an ongoing maintenance tax, the correct engineering choice is to clobber it.
The Theoretical Promise vs. Production Reality
The original design thesis behind Wikantik's Knowledge Graph was inspired by the broader industry enthusiasm for “GraphRAG” in 2024. The hypothesis held that while dense vector embeddings capture fuzzy semantic proximity across text chunks, they struggle with precise relational reasoning (such as identifying all components dependent on a specific subsystem, or tracing multi-hop corporate hierarchies). By introducing an automated pipeline to extract named entities (wk:Person, wk:Organization, wk:SoftwareSystem) and directed predicates (wk:dependsOn, wk:maintains, wk:authorizes), the system was expected to build a queryable semantic mesh that would elevate search relevance and allow LLM agents to execute multi-hop reasoning over the wiki corpus.
To execute this thesis, Wikantik built a comprehensive semantic web infrastructure:
- 15 Dedicated Relational Tables: Staging tables for raw extracted mentions (
chunk_entity_mentions), candidate node deduplication (kg_nodes), directional relations (kg_edges), an asynchronous curation workflow (kg_proposals), and graph clustering partitions (hub_members,hub_proposals). - LLM Extraction & Evaluation Pipeline: Asynchronous batch workers querying local Ollama or cloud models to parse document chunks into entity-relation triplets, followed by a secondary LLM “judge” tasked with scoring proposal validity and rejecting hallucinations.
- Ontology Governance: Strict SHACL (Shapes Constraint Language) validation rules enforcing type safety, domain/range constraints, and preventing ontological drift across Apache Jena in-memory models.
In controlled synthetic benchmarks with tiny toy datasets, this architecture appeared promising. In live production against a diverse, evolving corpus of technical documentation, architectural decision records, and operational runbooks, the cracks immediately widened into systemic failures.
Failure Mode 1: Entity Resolution and the Graph Noise Explosion
The primary algorithmic failure was entity resolution. Natural language documentation contains countless orthographic and contextual variants for the exact same underlying concept. A systems engineer might refer to “PostgreSQL” on one page, “Postgres DB” on another, “the relational datastore” in an ADR, and pg_dump in an operational script.
Because the extractor operated on local document chunks, it continually synthesized disparate entity nodes for identical targets. Attempting to solve this through automated string distance (Levenshtein, Jaro-Winkler) or secondary embedding clustering introduced catastrophic false positives: distinct concepts were erroneously merged, while identical systems remained fractured into disconnected subgraphs.
The resulting graph was simultaneously sparse where density was needed (missing true architectural links) and dense where noise was toxic (cross-linking unrelated concepts because an LLM hallucinated a generic wk:relatesTo predicate). When automated agents traversed these edges during multi-hop retrieval, they were routinely led down semantic detours that polluted the final context window with completely irrelevant text.
Failure Mode 2: Zero Empirical Uplift in Search Ranking
To evaluate whether the Knowledge Graph justified its presence, we instrumented systematic retrieval quality evaluations across our corpus, comparing three retrieval strategies:
- Dense Vector Retrieval: Cosine similarity against 768-dimensional chunk embeddings indexed via
pgvectorusing Hierarchical Navigable Small World (HNSW) indexes. - Page Graph Determinism: Ranking pages based on authored bidirectional wikilinks (
[PageName]), parent-child namespaces, and explicit citation structures. - GraphRAG Traversal: Expanding query candidates across 1-hop and 2-hop extracted entity paths weighted by graph centrality (PageRank on
kg_edges).
| Retrieval Pipeline | Mean Reciprocal Rank (MRR) | NDCG@10 | P95 Query Latency | Ingestion Write Overhead |
|---|---|---|---|---|
| Dense pgvector + BM25 (Hybrid) | 0.842 | 0.887 | 42 ms | < 350 ms / page |
| Deterministic Page Graph Only | 0.781 | 0.814 | 12 ms | < 20 ms / page |
| Knowledge Graph Traversal (1-Hop) | 0.694 | 0.723 | 285 ms | 4,200 ms / page |
| Hybrid + Knowledge Graph Expansion | 0.818 | 0.841 | 340 ms | 4,200 ms / page |
The empirical results were damning. Not only did Knowledge Graph traversal fail to outperform hybrid dense retrieval; activating graph expansion actually reduced Mean Reciprocal Rank from 0.842 down to 0.818. Spurious edges introduced irrelevant context chunks into the top-10 result set, diluting context density for downstream reasoning models. Furthermore, running multi-hop relational joins across relational tables ballooned query latency by more than 700%.
Every architectural subsystem must pay rent in the form of measurable user value, reduced operational cost, or superior output quality. If an automated subsystem consumes 80% of ingestion processing time, burns millions of inference tokens, and introduces 15 database tables, but yields a negative delta on retrieval accuracy, it must be removed without nostalgia.
Failure Mode 3: The Operational Ingestion Tax
Beyond search ranking failure, the operational burden of maintaining the Knowledge Graph was crippling to system throughput:
- Ingestion Wall-Clock Bottlenecks: When an engineer or autonomous agent edited a 2,000-word page, generating vector embeddings took roughly 300 milliseconds. In contrast, running the chunk extractor, generating candidate entity triplets, executing the LLM judge, and validating SHACL constraints took between 4 and 8 seconds per page. During batch document imports, the extraction worker queue backed up for hours.
- Database Lock Contention & Migration Risk: Maintaining relational foreign keys across 15 interconnected tables caused frequent row locks during high-concurrency automated agent runs. Upgrading schema definitions required complex multi-step data migrations that risked data integrity during crash recovery.
- Cognitive Overhead for Operators: Wiki administrators were forced to navigate proposal curation queues, review rejected relationship types, and manage vocabulary drift across an ontology that provided no tangible benefit to readers.
The Execution: How We Clobbered the Subsystem
To safely decommission the Knowledge Graph without risking data corruption or service downtime, we executed a phased two-step removal across releases 2.5.12 and 2.5.13:
Step 1 (Release 2.5.12): Fencing and Dark-Launch Retirement
In commit 7c99136a08, the configuration property wikantik.knowledge.enabled was switched to default to false. When disabled, the entire ingestion pipeline bypassed entity extraction, and ontology rebuilds were restricted to document classes (wk:Article) and SKOS concept schemes.
All reader-facing /knowledge-graph UI routes and admin curation panels were stripped from the web application. Simultaneously, an ArchUnit architectural rule (KnowledgeGraphFenceArchTest) was frozen in CI/CD, guaranteeing that no existing or newly written code in the core wiki runtime could introduce new dependencies on the quarantined knowledge graph packages.
Step 2 (Release 2.5.13): Total Physical Deletion
With production running smoothly for multiple days with zero reads against the graph tables, we executed the definitive deletion pass in commit 1b7ec8b6cc:
- Database Table Archival & Drop: We executed a full
pg_dumpof the 15 legacy graph tables to secure cold backup storage. Flyway migrationV070__drop_knowledge_graph.sqlthen executedDROP TABLEcascades againstkg_nodes,kg_edges,kg_proposals,chunk_entity_mentions, and all associated hub tables. - Codebase Purge: We deleted all 154 production Java files and 106 test classes associated with extraction, curation, judging, and graph REST resources. All
bin/kg-*administrative shell scripts were removed from disk. - Ontology Trimming: The ontology was streamlined strictly to document structural classes and topic concepts. The 9 entity classes, 21 relationship predicates, and SHACL constraint checkers were dropped, leaving
get_ontologyfast, deterministic, and lightweight.
What Remains: High-Density Primitives That Actually Work
Crucially, retiring the Knowledge Graph did not degrade Wikantik's semantic capabilities. In fact, retrieval is now substantially faster, cleaner, and more accurate because the system focuses exclusively on high-signal primitives:
- Dense Chunk Retrieval: Tables
kg_content_chunksandcontent_chunk_embeddingsremain fully active. Text chunks are embedded via lightweight local or cloud embedding models intopgvector, delivering sub-50ms semantic search that handles phrasing nuances, synonyms, and user intent effortlessly. - The Deterministic Page Graph: The true knowledge graph of an organization is not synthesized by an LLM guessing at triplets; it is authored deliberately by humans and agents through explicit internal links (
[PageName]), category tags, and formal citations. Wikantik's Page Graph remains an instant-access, deterministic structural map that accurately represents document relationships without hallucinations—see Page Graph vs. Knowledge Graph and the Hybrid Search Architecture reference on the wiki. - Streamlined Agent MCP Context: Autonomous coding agents interacting with Wikantik via the Model Context Protocol now query
retrieve_context, receiving high-relevance, deduplicated text bundles and citations without wasting tokens navigating synthetic relationship graphs.
Conclusion: The Value of Negative Work
In software engineering, there is a natural bias toward addition—adding new models, adding layers of abstraction, adding graph databases, and accumulating architectural weight under the assumption that more complex pipelines produce smarter systems. Wikantik 2.5 stands as a testament to the opposite discipline.
By removing 70,500 lines of unproven complexity, dropping 15 database tables, and cutting ingestion latency by 90%, we made Wikantik significantly faster, dramatically more reliable, and mathematically more accurate. The future of agent-grade knowledge retrieval belongs to fast, dense embeddings anchored in deterministic human structure—not brittle, synthetic web graphs.
Comments