OKF Agent Memory – Git-native persistent memory for AI coding agents (github.com)

79 pointsby okf_memory4 days ago32 comments

langs 3 days ago

I don't get it. Why benchmark the latency instead of recall/precision? Optimizing for millisecond-level latency is meaningless in the context of LLM calls. Accuracy is the tool's greatest value, yet there is no testing for it?

esafak 3 days ago

Has anyone else benchmarked all these tools for precision/recall? I too want to know if agent memory is something I should add. I only do session memory for now and that is quite useful.

glub 2 days ago

Yes. See LongMemEval, LoCoMo. Tons of research here.

But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.

We're now in the phase of re-discovering 30+ years of pain of knowledgebases.

olenzma a day ago

Invalidation is not an algorithm, but a judgment made in context

vshulcz 2 days ago

Benched cold start recently: 19k LongMemEval sessions laid down in the real ~/.claude and ~/.codex layouts, 100 questions whose answer sits in exactly one session, scored by whether that session comes back (bias: I built deja, one of the six)

deja: 29s to index, 24ms query, 18/100 hit@1, 67 found@50. Plain BM25, no vectors

agentmemory: 95s import, 14 hit@1, 65 found@50, plus a worker and engine on four ports

MemPalace: ~3h mining, 2.6s query, 14 hit@1

CASS: 56m index, every NL query fails with "query fuel exhausted" on the release build (fixed on their main)

claude-mem: no-op out of the box, only records forward from install

funes: the documented 1 min first pass indexed 189 of 19k sessions (0/100); full index still embedding, ~3 sessions/s

Numbers look low because 19k sessions is brutal; on the standard 500-session LongMemEval-S the same BM25 gets ~85% hit@1

The funny thing is BM25 basically ties embeddings here at 1/100th the cost. The real cliff is reranking (found@50 67 vs hit@5 35) and staleness. Vector search has zero concept of "superseded info" only fix I found was letting explicit user corrections outrank the transcript.

Repro scripts and corpus: https://vshulcz.github.io/deja-vu/guide/day-zero.html

@skeledrew: cross-agent across 23 harnesses, but yeah, it's an index over logs, not a source of truth :)

opwizardx 2 days ago

I did similar tests on my own corpus when was considering whether to keep semantic search in default path for pond.

On 3 months of my own sessions I’ve seen that BM25 search was finding the correct answer in ~61%, where semantic had shown only ~37% of success.

After that it was easy for me to make the decision.

Got all info on how I did evals in here, if interested: https://github.com/tenequm/pond/tree/main/docs/researches/26...

kimseungyong 2 days ago

I like the idea.

To avoid losing context, I mainly conduct the planning session and the implementation session separately.

From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.

If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.

However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.

iJohnDoe 3 days ago

This one was posted before?

https://github.com/fellowgeek/mcp-memory

https://news.ycombinator.com/item?id=49286073

straumat 2 days ago

Nice tool! If you want to publish your okf bundles for humans to read (like on a github pages or your intranet), i developped an open source solution to do that : https://github.com/oak-invest/kiso - It's like Hugo for OKF

calebkaiser 3 days ago

Love seeing projects like this. The performance benchmarks are nice to see. Have you done any benchmarks against approaches like OpenAI's Symphony for things like token usage or task completion?

rogeliodh 3 days ago

How does it compare to https://github.com/scaccogatto/okf-skills ?

swordsith 3 days ago

Personally I've always seen AI 'memory' as a pain point for people in their experience using LLMs than a benefit from the agent remembering the last unrelated thing you were working on. It wastes context similarly to 'skills'. The most efficient workflow imo is having a few well written (not by ai) md files across a clean codebase.

skeledrew 3 days ago

I want this, but also for cross-project memory. Save me from building my own, which I have planned but figure something would eventually pop up in HN...

techgnosis 3 days ago

https://github.com/huggingface/funes

Maybe?

skeledrew 3 days ago

Not really. Could be a starter, but I specifically want everything to be stored as plain text and no vector search.

opwizardx 2 days ago

plain text + fts covers all the basic needs, adding vector search, in my experience, adds more hassle and failures then helps

_ink_ 3 days ago

Or https://github.com/mempalace/mempalace

vshulcz 2 days ago

https://github.com/vshulcz/deja-vu

opwizardx 2 days ago

I had been building https://github.com/tenequm/pond for this exact purpose for a while, and it works like a charm in my case.

Give it a try, hope it will help you to solve your need without injecting anything in your context all the time.

practicalsystem 3 days ago

cool going to check it out, I've got an opensource project that might compliment it that I'm excited to try

https://github.com/ucsandman/declick

triyambakam 3 days ago

How well does the model adhere to using this in a harness like Codex where it may be directed to use the built in memory tooling? Maybe I'll need to try an experiment directing it to save to its native memory to use OKF instead

mbreese 3 days ago

This is usually my main concern with tooling like this that isn’t a first party project. Anthropic can tune Opus, Fable, etc and their harness to use their memory format or preferred method of tool calling. I have had mixed results getting LLMs to consistently use third party tools.

I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).

steammaho 3 days ago

honestly didn't notice much degrading of standard compact. Have several sessions which already lasts for several month and they are perfectly fine

opwizardx 2 days ago

Why do you need an MCP server at all when the spec itself is so simple?

hankbond 4 days ago

How are you using this compared to more explicit approaches where you lay out the project documentation in certain formats and conventions?

nullbio 3 days ago

Now that Astra is moving to a new compaction model (aka, ditching compaction altogether), is there any need for this sort of thing still?

pdimitar 3 days ago

Huh? Can you show source on the "it will not compact"? Very interesting.

esafak 3 days ago

"With Astra, we’re introducing a new way for Codex to preserve and retrieve context when the context window fills. Historically, models have used compaction to summarize work during long sessions, such as when debugging complex issues or tackling large refactors. Each compaction can leave out details about why a fix failed or how a component behaves. In Codex, Astra can keep notes across context windows, preserving accumulated details without repeatedly compressing them into a single summary. Earlier context windows remain searchable, so Astra can find requirements or test results from previous messages and tool outputs—even if that information wasn’t captured in its notes. You can enable this experimental feature in your Codex config.toml, (opens in a new window) and it will become the default for Astra in the coming weeks."

https://openai.com/index/gpt-6-astra/

bayesianbot 3 days ago

This clearly is Codex-specific, not as much a feature of the model (though obviously they probably have trained it to be great at working with their own tools). Sounds somewhat similar to pi-observational-memory I'm using with Pi.

svyatov 3 days ago

The performance is cool and all, but what about capture/retrieval quality? Are there any benchmarks for that?

okf_memory 4 days ago

Hey HN,

We built OKF Agent Memory because we were frustrated with how AI coding agents (Claude Code, Cursor, Windsurf, local models) handle long-term project context.

Every time a context window closes or a session resets, the agent forgets architectural decisions, domain discoveries, and operational rules. The existing solutions fall into two extremes: 1. Ad-hoc flat files (CLAUDE.md, AGENTS.md, .cursorrules) that inevitably balloon into 20k-token monoliths, degrade agent focus, and cause "lost-in-the-middle" attention failure. 2. Vector databases / background daemons (Mem0, Letta, Zep) that introduce heavy runtimes (Python/Node), docker containers, proprietary storage silos, and recurring embedding API costs (adding 200–800ms per retrieval call).

Our approach: The "LLM Wiki" in pure Go.

OKF Agent Memory (v0.1.0) is a single, zero-dependency Go binary that turns your Git repository into a structured, self-validating knowledge corpus based on Google's Open Knowledge Format (OKF) v0.2 specification:

• In-Memory BM25 Search (<300µs): Fast lexical ranking across titles, YAML metadata, tags, and bodies directly in memory. No embedding APIs, zero network overhead, zero runtime cost. • Progressive Disclosure: Slashes prompt overhead by up to 90%. Instead of loading thousands of lines of context, the agent searches the bundle index and pulls only the exact 300-token concept required for the current task. • 100% Git-Native: Everything lives in `knowledge/` as human-readable Markdown. You audit your agent's memory via `git diff`, `git blame`, and code reviews. • Built-In Stdio MCP Server: `okf mcp knowledge` exposes native Model Context Protocol tools (`okf_search`, `okf_show`, `okf_create`, `okf_validate`) directly to Claude Code and Cursor. • Trust Tiers: Distinguishes authoritative human law (`verified: human:...`) from agent-generated drafts (`generated: agent:...`). • Sub-4ms Cold Starts (<15MB RSS): Starts in milliseconds with no VM spin-up.

Try it in 30 seconds: $ brew install okf-memory/tap/okf $ cd your-project && okf bootstrap .

GitHub: https://github.com/okf-memory/okf-agent-memory Docs & Landing Page: https://okf-memory.dev

We'd love your feedback on the architecture, the Go implementation, and how your coding agents behave with progressive disclosure memory!

lukevp 3 days ago

I love the idea! OKF 0.2 solves for the problem of the AIs generating massive amounts of documentation (far more than humans ever created) and giving it equal-weight over what a human actually approved and committed to. I've been using it to attribute my decisions with specific directions on how to ensure that it takes strong direction from my explicit decisions and clarifies implicit / AI-driven decisioning.

Adding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.

One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?

If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.