©2026 adebench · adecubedbuilt 2026-11-04 from the reports in the repo

← leaderboard

agentmemory (Gemini)

the same memory with its LLM features on: compression, graph extraction, consolidation. https://github.com/rohitg00/agentmemory

Core
65% · 35.5 / 55 (mean)
Core (public2)
public2 run in progress
Private set
after the public2 runs
Full
66% · 42.6 / 65 · 65 of 100 measured
Runs
5 · core 35.5–35.5 · the details below are the median run
Common with gbrain
42.6 vs 50.4 on 65
Chars per answer
6,338 · answer after 362
Write → visible
253 ms · p95 300 ms
Write-back
0 of 0 degraded answers came back
Reached through
MCP / REST
Model on write
yes: compression and graph extraction (Gemini)
Key needed
LLM provider
Where it runs
local
License
Apache-2.0
Deterministic
yes in five builds
Configuration
Version@agentmemory/agentmemory 0.9.29 (npm) with iii-engine 0.11.2
Modewith Gemini: LLM features on (compression, graph extraction, consolidation)
Doormemory_recall(query, limit=10) through POST /agentmemory/mcp/call, the route the MCP stdio shim forwards to; the tool's own text
Resultslimit 10 (its default), format full (its default), no relevance floor, no cut
Model insidegemini-3-flash-preview (Google): compresses observations, extracts the knowledge graph, runs consolidation
Embeddingsgemini-embedding-001, 768 dimensions (Google, the one Gemini embedder it ships)
Storagelocal scratch data directory on the iii engine (format not stated)
Not defaultGEMINI_MODEL=gemini-3-flash-preview and EMBEDDING_PROVIDER=gemini (the Gemini setup); AGENTMEMORY_AUTO_COMPRESS=true, GRAPH_EXTRACTION_ENABLED=true, CONSOLIDATION_ENABLED=true (the three flags /config/flags lists as needing an LLM); the import also runs /graph/build and /consolidate-pipeline

Notes: npm 0.9.29, gemini-3-flash-preview, gemini-embedding-001; same door and set as the keyless row. Run 2026-09-30, door recall, fingerprint 34483ae0a3. Report: examples/agentmemory_report/reference_gemini.md · All runs: examples/agentmemory_report/repeats_gemini.json

Sections
door
18.8
of 25 · 6 pass · 2 fail
FAIL Which version of MailBridge is installed for the mailbox? — STALE value delivered next to the current one: 1.3.0
FAIL What is the owner's phone number? — missing +39
cards
not measuredsection not measured: no entity cards in this memory
updates
6.7
of 10 · 2 pass · 1 fail
FAIL restating the current value does not pile up a second copy — 3 copies delivered
time
6.7
of 10 · 4 pass · 2 fail
FAIL semantic memories carry their age (38/49) — 78%
FAIL an imported memory reaches the door with its original date — served with no date
live state
7.1
of 10 · 5 pass · 2 fail
FAIL after overwriting the canary the door serves the new value, never the old one — overwrite-to-visible 315 ms · STALE: old and new value delivered together
FAIL two writes in quick succession: the door settles on the second, never back on the first — second write visible in 270 ms · 24 read(s) served the first write after the second was already visible
abstention
3.3
of 10 · 0 pass · 4 fail
FAIL What is the Zarpetta module? — episodes presented as direct matches; 6 keyword facts for something that does not exist
FAIL Who is Ottavio Brambillesco? — episodes presented as direct matches; 6 keyword facts for something that does not exist
FAIL Which port does the Fulmicotone service use? — episodes presented as direct matches; 6 keyword facts for something that does not exist
FAIL What does the calendar plugin Girandola do? — episodes presented as direct matches; 6 keyword facts for something that does not exist
file search
not measuredsection not measured: no repo configured (--repo)
graph
not measuredsection not measured: no case with evidence (all SKIP or no cases)