©2026 adebench · adecubedbuilt 2026-09-30 from the reports in the repo

← leaderboard

supermemory

memory and context engine: an LLM agent extracts and versions memory entries from documents. https://github.com/supermemoryai/supermemory

Core
69% · 38.1 / 55
Full
74% · 48.1 / 65 · 65 of 100 measured
Common with gbrain
48.1 vs 50.4 on 65
Chars per answer
364 · answer after 81
Write → visible
2.1 s · p95 2.1 s
Write-back
0 of 2 degraded answers came back
Reached through
REST
Model on write
yes: an LLM agent extracts every document
Key needed
LLM provider
Runs
self-hosted
License
MIT
Deterministic
no: extraction is a model call

Configuration: self-hosted server 0.0.8, native Gemini provider (gemini-3.1-flash-lite-preview, fixed by the binary), local bge-base-en-v1.5; reference = median of three builds (48.1, 48.1, 52.1). Run 2026-09-30, door memories, fingerprint e37cca2b5e. Report: examples/supermemory_report/reference.md

Sections
door
15.6
of 25 · 5 pass · 3 fail
FAIL Which version of MailBridge is installed for the mailbox? — STALE value delivered next to the current one: 1.3.0
FAIL Where does the owner live? — missing Turin
FAIL What is the owner's phone number? — missing +39
cards
not measuredsection not measured: no entity cards in this memory
updates
10.0
of 10 · 3 pass · 0 fail
time
5.0
of 10 · 3 pass · 3 fail
FAIL semantic memories carry their age (37/45) — 82%
FAIL an imported memory reaches the door with its original date — imported memory not served within 30 s
FAIL «what did pc2 do?» finds the episodes signed [pc2] — 0 episodes in the answer
live state
10.0
of 10 · 7 pass · 0 fail
abstention
7.5
of 10 · 2 pass · 2 fail
FAIL Which port does the Fulmicotone service use? — 3 keyword facts for something that does not exist
FAIL What does the calendar plugin Girandola do? — episodes presented as direct matches; 6 keyword facts for something that does not exist
file search
not measuredsection not measured: no repo configured (--repo)
graph
not measuredsection not measured: no case with evidence (all SKIP or no cases)