memoket-kite is an open-source long-term memory engine for AI agents that eliminates the embedding stack entirely and reports state-of-the-art results on the LoCoMo benchmark. The case against vector-based memory at session scale is stronger than most engineers are willing to admit. The open question is whether benchmark performance on curated English conversational data transfers to the adversarial, multilingual, domain-specific conditions of production deployments — and current evidence does not confirm it does.
The engineering default for agent memory is a vector store. You embed the conversation, chunk it at fixed token boundaries, index it in Pinecone or Chroma or Weaviate, and retrieve by cosine similarity at inference time. This architecture was designed for retrieval-augmented generation over static document corpora. It is being retrofitted onto long-term agent memory, and the seams are showing.
Vector-based retrieval has three compounding failure modes at session scale. First, semantic drift: cosine similarity assumes embedding space is stable, but conversations shift register, introduce new entities, and reuse the same surface forms with different meanings across sessions. "Model" means something different in a conversation about machine learning than in a conversation about business strategy. An embedding model trained on general text handles this ambiguity poorly when the sessions are months apart. Second, chunk fragmentation: splitting long conversations at fixed token boundaries destroys the sequential context that gives individual facts meaning. A statement that was meaningful in session four becomes noise when severed from the exchange that preceded it. Third, retrieval threshold brittleness: the cosine cutoff that returns relevant memories cleanly in the early sessions floods the context with low-quality matches as the index accumulates thousands of entries.
memoket-kite discards the embedding layer entirely. Based on the project repository, the engine represents long-term memory in structured form — entities, attributes, relationships, and temporal context stored in a queryable format without similarity search. Retrieval is deterministic. The system returns what you ask for without nearest-neighbor noise.
This is not a new architectural idea. Knowledge graphs and structured memory representations predate LLMs by decades. What makes memoket-kite worth examining seriously is the combination of benchmark evidence and the engineering discipline to ship it as a drop-in layer rather than a research artifact.
Most RAG evaluations measure single-turn retrieval precision over static document corpora. A query comes in, relevant chunks are retrieved, and the system either returns the right passage or it does not. The evaluation is clean, repeatable, and largely disconnected from what agent memory actually looks like in deployment.
An agent handling a month of user interactions has accumulated thousands of turns across dozens of sessions. The embedding index grows. Retrieval latency compounds. The probability of a false positive — a "relevant" memory from a different context window or a mismatched entity — increases with index size in ways that single-session benchmarks do not expose.
LoCoMo was designed specifically to surface this failure. The benchmark, published at ACL 2024, constructs evaluation instances that span multiple conversation sessions, requiring memory systems to answer questions about facts stated sessions earlier. Standard vector retrieval systems show consistent performance degradation as session depth increases. memoket-kite's structured representation sidesteps the degradation by not relying on similarity thresholds that drift with index growth.
LoCoMo is the most credible public benchmark currently available for long-term conversational memory, and it has legitimate methodology. The evaluation is QA-based: given a question about a fact from earlier in a long conversation chain, the system must produce the correct answer. Difficulty scales with session depth. Questions targeting facts from early sessions require genuine long-term retention, not just short-context look-ahead.
memoket-kite reports SOTA on this benchmark. Take that at face value, but understand precisely what it means.
LoCoMo covers English social conversations — episodic, personal dialogue about life events, relationships, and preferences. The facts embedded in these conversations are discrete and entity-attributable: "you mentioned you started a new job last month," "your sister lives in Seattle," "you said you were training for a marathon." These facts have a natural affinity with structured representation. They map cleanly onto entity-attribute tuples. A knowledge store retrieves them exactly. A vector index retrieves them noisily.
The benchmark is well-constructed for what it measures. The issue is the scope of what it measures.
Three gaps are worth naming explicitly.
Technical and domain-specific vocabulary is not tested. LoCoMo uses everyday language. An agent embedded in a codebase, a medical record system, or a financial workflow deals with terminology where general embedding models fail. "Attention" means something different in an ML engineering context than in a general conversation. Structured entity extraction may actually perform better in technical domains — exact match on precise terminology beats fuzzy similarity over ambiguous embeddings — but the benchmark does not confirm this.
Adversarial memory conflicts are absent from the evaluation protocol. Production agent memory systems will encounter contradictory information: users who correct earlier statements, change stated preferences, or provide conflicting facts across sessions. How the consolidation logic handles overwrites and retractions is not characterized by LoCoMo, and it matters more than benchmark score for deployed systems.
Scale beyond benchmark bounds is uncharacterized. LoCoMo evaluation instances are bounded by design. A deployed agent accumulating memory over months operates in a different regime entirely. What happens to structured retrieval at 50,000 entities? At 500,000? The benchmark does not reach this regime, and no public evaluation currently does.
If you are building long-term memory for an agent that needs to reason across sessions, memoket-kite offers something the vector-store ecosystem does not: deterministic retrieval with inspectable logic. You know exactly why a memory was returned or was not.
This matters in two practical contexts.
First, debugging. When an agent fails to recall a critical fact from a prior session, you need to know whether the failure was in retrieval (the correct memory was not returned), in storage (the fact was never consolidated), or in reasoning (the retrieved memory was misinterpreted). Vector-based systems make this diagnosis difficult because retrieval is probabilistic and opaque. Structured retrieval makes the failure point explicit and traceable.
Second, compliance. Applications where memory content has regulatory implications — healthcare records, financial advice, legal assistance — require memory systems that can explain what they store, retrieve, and discard. Cosine similarity does not have an explanation layer. Structured memory logic does.
For teams building agents with persistent state across long-running tasks — the class of architecture represented by frameworks like the open-source AI coworker infrastructure built on CopilotKit/OpenBot, where agents need browser access, file access, and continuity across sessions — memoket-kite is the kind of memory layer that makes the persistence meaningful rather than approximate.
The no-fine-tuning, no-embedding-model design has a direct operational consequence: memoket-kite runs on top of whatever model you are already running, including quantized variants. If you are running a 4-bit GGUF model on a single GPU or a local machine, you do not need a separate embedding model consuming additional VRAM. The memory engine does not interact with model weights at all.
Most embedding-based memory stacks require a separate embedding inference endpoint — either hosted or self-hosted. memoket-kite eliminates that dependency entirely, which simplifies the infrastructure stack and removes a latency hop from the retrieval path.
Don't use it for semantic similarity search over unstructured documents. If your agent needs to find documents or passages that are thematically related to a query without exact entity matches, structured memory without embeddings cannot help you. That is a RAG use case, not a long-term agent memory use case. The distinction is architectural, not a matter of configuration.
Don't deploy in multilingual contexts without evaluation. LoCoMo is English-only. Structured entity extraction degrades in multilingual settings unless the extraction logic is explicitly multilingual-aware. The benchmark provides no signal about cross-lingual memory performance.
Don't assume the consolidation logic is correct for your domain. The most consequential and least visible component of any long-term memory system is the logic that decides what to keep, what to overwrite, and what to discard. In memoket-kite this logic is inspectable — that is a genuine advantage. But "inspectable" does not mean "correct for your use case." Read the consolidation code before deploying in contexts where forgetting the wrong fact has real consequences.
Don't treat SOTA on one benchmark as production readiness. This applies to every memory system, not just this one. LoCoMo is a rigorous benchmark. It is not a complete characterization of the failure modes you will encounter.
Structured memory becomes a first-class agent primitive. The current dominant architecture treats memory as a retrieval problem — find the relevant past context and inject it. The more precise framing is memory as a maintained knowledge base: a structured world model over the agent's interaction history that supports explicit queries, updates, and conflict resolution. memoket-kite is an early implementation of this framing. The trajectory is clear.
Benchmark methodology will improve, and rankings will shuffle. LoCoMo is a significant step forward from earlier memory evaluation protocols. The next generation of benchmarks will include adversarial memory conflicts, temporal reasoning over updated facts, and cross-domain transfer. When those exist, current SOTA rankings are likely to change. Treat them as directional signals, not definitive orderings.
The vector-free argument strengthens as context windows grow. There is a structural paradox worth tracking: as base model context windows expand to 200k, 1M, and beyond, the case for external memory weakens for moderate-length interactions. But for genuinely long-running agents accumulating months of history, no context window will be sufficient, and structured persistent memory becomes the only viable option. The engineering debate will shift from whether to use external memory to how it should be structured and what consistency guarantees it should provide.
Reasoning chains gain explicit memory query operations. Memory systems that expose structured representations can be queried by name in reasoning chains. A chain-of-thought that executes query_memory(entity="project_deadline", attribute="date") is verifiably different from one that injects a block of loosely similar text retrieved by cosine score. Structured memory enables reasoning steps that are explicit, auditable, and wrong in diagnosable ways rather than opaque in untraceable ones.
Privacy-sensitive deployments consolidate around open stacks. The combination of open-source LLMs — Llama, Mistral, Gemma, Qwen — with open-source memory engines like memoket-kite creates a self-contained agent stack with no external API dependencies. For air-gapped environments, on-premise healthcare deployments, and any context where sending memory content to a cloud embedding endpoint is not viable, this combination matters practically.
How does memoket-kite handle memory conflicts when a user corrects an earlier stated fact?
The consolidation logic handles this, and it is inspectable in the codebase. That said, conflict resolution in long-term memory is an unsolved research problem, and LoCoMo does not include adversarial correction scenarios in its evaluation. The system's behavior under contradiction is not benchmarked. Test this explicitly before deploying in domains where users frequently update or retract prior statements.
Does the vector-free approach reduce retrieval quality for fuzzy or paraphrased queries?
Almost certainly in some cases. Embedding-based retrieval handles paraphrase and synonym variation more gracefully than structured exact-match retrieval. The trade-off memoket-kite makes is deliberate: sacrifice some fuzziness tolerance to gain deterministic retrieval and eliminate similarity drift. On LoCoMo — where facts are discrete and entity-attributable — that trade-off is justified by the benchmark results. On use cases requiring semantic generalization over ambiguous queries, evaluate carefully before committing.
What happens to retrieval performance at large memory scales?
Not publicly characterized at the time of writing. Structured retrieval can maintain sub-linear lookup times with appropriate indexing, but performance at large memory stores — tens of thousands to hundreds of thousands of entities — depends on the storage backend and query patterns. This is a gap in the published characterization. Test empirically against your expected memory volume before production commitment.
Can it run on top of quantized models?
Yes. The memory engine is decoupled from model weights entirely. Quantization of the base LLM does not affect memory storage or retrieval mechanics. What quantization affects is the quality of entity extraction during memory consolidation — a noisier base model produces noisier entity extractions, which end up in the memory store. That is a base model quality issue, not a memory engine issue, but it surfaces in memory system output.
How does it compare to MemGPT or Letta's hierarchical memory approach?
MemGPT uses a hierarchical memory architecture with LLM-managed memory operations and vector retrieval for external storage. The explicit comparison on LoCoMo would be informative; the memoket-kite benchmark results should be read against the specific baselines the evaluation reports. Do not assume benchmark rankings transfer to your specific query distribution. Run both on your actual data.
Does the absence of an embedding model reduce infrastructure cost meaningfully?
Depends on scale. For a single developer or small team, the cost of an embedding endpoint is negligible. For applications running millions of memory consolidation operations per day, eliminating the embedding inference call removes a real cost and latency component. The operational simplification — no embedding model to manage, version, or monitor — has value independent of direct cost.
Is there a risk that "vector-free" becomes a marketing label without architectural substance?
Yes, and it is worth verifying in the codebase. "Vector-free" should mean no embedding model and no similarity-based retrieval, not just no visible vector database dependency. The distinction matters because some approaches use embeddings internally without exposing them through a standard vector store interface. Read the actual retrieval path in the code before accepting the label.
![Open-source single-GPU reproductions of Cartridges and STILL for neural KV-cache compaction [P] image](/_next/image?url=https%3A%2F%2Fbestaifor.s3.eu-west-1.amazonaws.com%2Fopen_source_single_gpu_reproductions_of_cartridges_featured_954b9a265f.jpg&w=3840&q=75)
Neural KV-cache compaction — using learned compression rather than heuristic eviction — is one of the more credible paths to running long-context LLMs without bleeding GPU memory. Cartridges and STILL are two recent...
Frontier model benchmark scores climbed between 8 and 45 percentage points across MMLU, SWE-bench, and ARC-AGI over the past 12 months. Workflow impact is real but concentrated — reasoning tasks and agentic code...
SWE-bench scores for top coding agents now clear 70%. Inside Databricks' production codebase—millions of lines spanning distributed systems, Scala runtime code, and internal APIs built over a decade—those numbers...
BestAIFor