Skip to content

Wikantik

The Wikantik Reader's Field Guide: How Storage, Hybrid Search, and Dual MCP Fit Together

From Wikantik · 22 articles · latest

Diagram linking POSIX Markdown storage, BM25 plus pgvector hybrid retrieval, and dual read/admin MCP servers
Wikantik's settled architecture: Git-backed Markdown as canonical truth, hybrid RRF search in PostgreSQL, and physically split read vs. write MCP endpoints.

If you land on Wikantik today, you are greeted by nineteen dense engineering postmortems and architecture notes spanning eight months of intense refactoring. You can read about a 79 GB Apache Jena TDB2 compaction leak, an N+1 database query storm, a 70,516-line Knowledge Graph deletion, and dual Model Context Protocol servers—but without a map, it is easy to miss how those pieces actually fit together into one working system.

This field guide is that map. Whether you are evaluating Wikantik as an agent-grade knowledge base or borrowing its architectural patterns for your own RAG stack, here is how the system settles together today. If you want the live reference documentation alongside these engineering notes, start with the Wikantik manifesto and architecture overview and the Wikantik Platform Hub on the wiki itself.

Layer 1: Canonical Storage Without Split-Brain

Wikantik began as a fork of Apache JSPWiki—which we modernized in From JSPWiki to Wikantik: The Jakarta EE 10 Migration—and moved its web tier onto a decoupled React SPA in Decoupling the Legacy Web Tier. Yet one core invariant survived every refactor: plain Markdown files on disk, backed by Git, remain the single source of truth.

Why refuse to move canonical page bodies into PostgreSQL? Because developers and coding agents already know how to diff, branch, grep, and back up Markdown files. When we audited storage edge cases in Silent Data Loss and Destructive Admin Tools, the fix was not abandoning files—it was enforcing POSIX atomic writes (write temp → fsync → rename) so neither crashes nor concurrent edits can corrupt a page.

Inside those Markdown files, we enforce strict Agent-Grade Content invariants: clean <h2>/<h3> structural spines, YAML frontmatter metadata, self-contained paragraph bundles with zero floating pronouns, and explicit temporal anchors.

Layer 2: Hybrid Retrieval (And Why We Deleted the Knowledge Graph)

Early on, we drank the GraphRAG Kool-Aid. We built an LLM entity extractor, 15 PostgreSQL ontology tables, SHACL validation, and graph reranking—and then measured it honestly. As documented in Evaluating GraphRAG and Why We Clobbered the Knowledge Graph, heuristic entity extraction degraded ranking precision while adding 70,516 lines of maintenance drag and a 79 GB Jena TDB2 disk leak.

So we deleted the entire extracted Knowledge Graph and doubled down on two deterministic pillars:

  1. BM25 + pgvector with Reciprocal Rank Fusion (RRF): Section-level chunking combines exact lexical precision (critical for class names, error codes, and config keys) with dense semantic cosine similarity inside PostgreSQL, merged via Reciprocal Rank Fusion.
  2. The Deterministic Page Graph: Explicit human- and agent-authored wikilinks ([Target](Target)) plus cluster tags form an unambiguous navigation graph without LLM hallucination—the clean separation explained in The Two Graphs.

Note

Honest Lesson From Production: When a complex subsystem makes your benchmarks worse and bloats your operational footprint, refactoring it gently is procrastination. Deleting 70,500 lines in one disciplined release (2.5.14) immediately freed engineering bandwidth for retrieval quality and observability.

Layer 3: Dual MCP Planes & Tiered Context Briefings

Agents interact with Wikantik through two physically separated endpoints implementing the Model Context Protocol, detailed in First-Class AI Citizens:

EndpointAudience & AuthPrimary ToolsPurpose
/knowledge-mcpRead-only agents & coding assistantsget_briefing, assemble_bundle, read_pages, get_page_neighborsGrounded retrieval within strict token budgets
/wikantik-admin-mcpAuthenticated admin/editor pipelinesPage writes, verify_pages, backlink & metadata auditsAtomic content mutations and graph integrity

The biggest ergonomic leap in Wikantik 2.5 was introducing get_briefing. Instead of burning 15 tool calls searching for architecture docs at the start of every coding session, an agent calls get_briefing once with its project cluster and initial task prompt, receiving its cluster pages, cited cross-repo contracts, and compact pointers within a single token budget (see the MCP Integration quickstart on the wiki for client setup and endpoint details). Follow-up questions route to assemble_bundle (which returns ranked, de-duplicated, version-pinned sections with exact URLs and confidence coverage) or read_pages (batched Markdown fetches capped at 20 slugs).

That separation matters in production because token budgets are real engineering constraints, as we measured in The Operational Reality of Agents in Production. Dumping entire 8,000-word architecture hubs into every session wastes context window capacity and dilutes attention on the actual code diff. Tiered briefings keep the agent grounded in cross-repo contracts without choking its reasoning budget.

Layer 4: The Cross-Repo Dogfood Loop

Wikantik is not a passive documentation archive sitting beside our code—it is the shared asynchronous bus coordinating jspwiki (Wikantik itself), pressroom, blogger-agent, stockroom, and jakemon, as laid out in The Agentic Dogfood Loop. Architectural Decision Records (SimpleAgilityDecisionLog) and initiatives (SimpleAgilityAgentCoordination) live in Wikantik, agents pull them via /knowledge-mcp before writing code, and production learnings—like eliminating N+1 query storms, taming God modules with a CI complexity ratchet, or mapping the unfinished horizontal scale roadmap—flow right back into the wiki.

If you are diving into the archive for the first time, skip chronological order and read these five posts as a spine:

  1. Beyond the Fork: Settling the Architecture
  2. Building the Hybrid Retrieval Engine: BM25 Meets pgvector
  3. Why We Clobbered the Knowledge Graph
  4. First-Class AI Citizens: Dual MCP Architecture
  5. The Operational Reality of Agents in Production

Next up in the stack is closing the feedback loop from live readers back into the knowledge base—connecting Google Search Console, cookieless Umami analytics, and jakemon so content improves itself from real reader signals.

Keep reading

Comments