By milestone 2.4.62, Wikantik addressed its primary runtime bottlenecks: the 79 GB Jena TDB2 disk leak was plugged, N+1 database statement amplification was collapsed, in-place file truncation was replaced with POSIX atomic writes, and dual Model Context Protocol gateways were stabilized.
However, a technical assessment of the platform requires examining its operational boundaries. Today, Wikantik operates exclusively as a single-host deployment. Understanding the long-term scalability of the platform requires identifying the hard constraints of this model and the architectural prerequisites for horizontal distribution.
The Single-Host Model: Current Operating Parameters
In production today, all components—the Apache Tomcat container, PostgreSQL 18 with pgvector, the embedded Apache Jena TDB2 ontology store, and the local Ollama embedding service—run co-located on a single physical host inside Docker.
For engineering organizations managing up to tens of thousands of documents and dozens of concurrent AI agents, the single-host architecture offers concrete operational advantages:
- Zero distributed network hop latency between storage tiers.
- Elimination of distributed consensus protocols (Raft, Paxos).
- Operational simplicity: backup strategies rely on standard filesystem snapshots and POSIX
pg_dumppipelines.
Under synthetic k6 load testing, the single-host system handles sustained throughput of 825 requests per second with a 95th-percentile response latency of 361 milliseconds. When sudden traffic surges exceed available thread workers, Wikantik’s BackpressureFilter sheds load deterministically using HTTP 503 and Retry-After: 1 headers, avoiding memory exhaustion.
Despite these characteristics, the single-host foundation encounters physical scaling limits as concurrent autonomous agent traffic increases.
Architectural Prerequisites for Horizontal Distribution
Distributing the Wikantik application tier across multiple compute nodes requires resolving three fundamental architectural dependencies:
1. PostgreSQL Connection Scaling and PgBouncer Integration
PostgreSQL allocates a dedicated backend OS process for each client connection. Consequently, memory overhead and context switching degrade throughput significantly when active connections exceed 100 to 200 sessions.
In a horizontally clustered deployment where multiple application nodes each maintain HikariCP connection pools of 30 to 50 connections, total database connections would quickly reach saturation.
The next major infrastructure milestone is introducing PgBouncer for transaction-level connection pooling. Positioned between the application nodes and PostgreSQL, PgBouncer multiplexes hundreds of incoming application client sessions across a pool of 20 to 30 persistent backend connections, allowing the compute tier to scale independently of PostgreSQL process limits.
The primary technical barrier to multi-node clustering is the embedded Apache Jena TDB2 ontology store. TDB2 is designed as an in-process, disk-backed engine governed by a local filesystem lock (tdb.lock). It cannot be mounted across Network Attached Storage (NFS/EFS) by multiple writer processes without risking B+tree index corruption.
Moving to a multi-node architecture requires one of two paths:
1. Migrating the ontology tier to an external, standalone client-server triplestore (such as Apache Jena Fuseki or OpenLink Virtuoso) communicating over HTTP and SPARQL 1.1 Protocol.
2. Maintaining read-only local TDB2 replicas on worker nodes, synchronized asynchronously from a primary write node.
2. Distributed Cache Invalidation
Currently, Wikantik maintains sub-millisecond read latency by caching document metadata, permission grants, and section anchor maps in local JVM memory via Caffeine caches.
In a multi-node topology, write operations on Node A must invalidate stale caches across Node B and Node C. Implementing distributed invalidation requires integrating a lightweight messaging bus (such as Redis or PostgreSQL's native LISTEN/NOTIFY mechanism) to broadcast cache eviction events across the application cluster.
Future Graph Research: Section-Level Reranking
Earlier in the platform's development, the team removed 2,464 lines of Knowledge Graph reranking code after empirical evaluation showed that document-level entity co-mention graphs produced zero net retrieval lift over hybrid BM25 + dense vector search.
However, analysis of those benchmark results demonstrated that the primary issue was granularity mismatch: applying graph boosts to entire multi-thousand-word pages distorted chunk selection envelopes.
An active research initiative explores section-level Knowledge Graph reranking:
- Entity mentions are mapped directly to specific 174-token content chunks rather than parent page containers.
- When a query references an entity, graph proximity boosts are applied strictly to the chunk asserting the specific relationship.
- Synthetic evaluations indicate that section-level scoping avoids high-degree hub distortion while preserving token budget efficiency. Production integration remains gated behind empirical validation of recall improvements.
Vertical Optimization and Infrastructure Discipline
In systems engineering, premature horizontal distribution frequently introduces operational complexity before single-node capacity is fully utilized.
Wikantik’s engineering roadmap adheres to a sequential strategy: optimize single-host resource utilization, profile storage and database bottlenecks directly, and establish modular boundaries before distributing compute across nodes.
By systematically eliminating N+1 queries, resolving storage compaction leaks, and enforcing POSIX atomic write durability, the system established a predictable runtime baseline. When the requirements for multi-node clustering emerge, the platform will transition to distributed topologies on top of hardened storage invariants.
Comments